2024-09-13T18:26:11.943 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T18:26:11.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:26:12.055 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993 branch: wip-rishabh-testing-20240910.100730-reef description: fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} email: null first_in_suite: false job_id: '7903993' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/quincy latest ' name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 targets: smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgtfPDrkS1U+OTKhEY3eqaaqlHpNjaFiq6uTr9JoH/BGmg6EX8v6uBA5+mVe5LIHo+u0H85+J6qv9fyCu17J+c= tasks: - install: branch: quincy exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing quincy' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client quincy - print: '**** done ceph' - print: '*** upgrading, no cephfs present' - exec: mon.a: - ceph fs dump - install.upgrade: mon.a: null - print: '**** done install.upgrade' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph versions - ceph osd dump -f json-pretty - ceph fs dump - ceph osd require-osd-release reef - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T18:26:12.055 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T18:26:12.056 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T18:26:12.056 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T18:26:12.058 INFO:teuthology.task.internal:Checking packages... 2024-09-13T18:26:12.079 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T18:26:12.079 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:26:12.080 INFO:teuthology.packaging:ref: None 2024-09-13T18:26:12.080 INFO:teuthology.packaging:tag: None 2024-09-13T18:26:12.080 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:26:12.080 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:26:12.080 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:26:12.234 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860.g6c6b083f 2024-09-13T18:26:12.236 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T18:26:12.243 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T18:26:12.243 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T18:26:12.248 INFO:teuthology.task.internal:Saving configuration 2024-09-13T18:26:12.265 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T18:26:12.271 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T18:26:12.303 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 18:21:21.160872', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgtfPDrkS1U+OTKhEY3eqaaqlHpNjaFiq6uTr9JoH/BGmg6EX8v6uBA5+mVe5LIHo+u0H85+J6qv9fyCu17J+c='} 2024-09-13T18:26:12.304 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T18:26:12.310 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-09-13T18:26:12.310 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T18:26:12.356 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f95a3483be0>, signals=[15]) 2024-09-13T18:26:12.356 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T18:26:12.363 INFO:teuthology.task.internal:Opening connections... 2024-09-13T18:26:12.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:26:12.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:26:12.451 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T18:26:12.458 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-09-13T18:26:12.535 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-09-13T18:26:12.536 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-09-13T18:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-09-13T18:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T18:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T18:26:12.595 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-09-13T18:26:12.625 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T18:26:12.635 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T18:26:12.640 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T18:26:12.641 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:26:12.658 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T18:26:12.665 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T18:26:12.665 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:26:12.803 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T18:26:12.848 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T18:26:12.865 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T18:26:12.865 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T18:26:12.865 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:26:12.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:26:12.883 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-09-13T18:26:12.940 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-503.el9.x86_64 2024-09-13T18:26:12.940 INFO:teuthology.task.kernel:Running kernel on smithi177: 5.14.0-503.el9.x86_64 2024-09-13T18:26:12.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2024-09-13T18:26:17.526 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-13T18:26:19.668 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-13T18:26:24.902 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-13T18:26:26.336 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 19 kB 00:00 2024-09-13T18:26:27.282 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-13T18:26:32.763 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T18:26:34.811 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T18:26:34.811 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T18:26:34.898 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:26:34.907 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:26:34.907 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:26:34.908 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:26:34.909 INFO:teuthology.orchestra.run.smithi177.stdout:Install 4 Packages 2024-09-13T18:26:34.909 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:26:34.909 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 79 M 2024-09-13T18:26:34.909 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 126 M 2024-09-13T18:26:34.909 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:26:35.106 INFO:teuthology.orchestra.run.smithi177.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 230 kB/s | 23 kB 00:00 2024-09-13T18:26:35.489 INFO:teuthology.orchestra.run.smithi177.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-13T18:26:35.856 INFO:teuthology.orchestra.run.smithi177.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-13T18:26:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-13T18:26:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:26:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:Total 63 MB/s | 79 MB 00:01 2024-09-13T18:26:36.846 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:26:37.190 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:26:37.191 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:26:38.163 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:26:38.163 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:26:40.610 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:26:41.259 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:26:41.341 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:26:44.145 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:26:44.277 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:26:49.141 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:26:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:27:32.053 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:27:32.093 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:27:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:27:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:27:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:27:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:27:33.278 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:27:33.278 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:27:33.278 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:27:33.278 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T18:27:33.278 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T18:27:33.278 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T18:27:33.279 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T18:27:33.279 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:27:33.279 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:27:33.639 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:27:33.734 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:27:33.735 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:27:33.735 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T18:27:33.735 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:27:33.735 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:27:33.735 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T18:27:33.735 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:27:33.735 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:27:33.795 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:27:33.795 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:27:33.795 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.06 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 82.el9 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Group : Unspecified 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 8274670 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T18:27:33.827 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : builder@centos.org 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:27:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T18:27:33.829 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T18:27:33.829 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 5.14.0-505.el9.x86_64 2024-09-13T18:27:33.830 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T18:27:34.443 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-09-13T18:27:35.828 INFO:teuthology.orchestra.run.smithi177.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T18:27:35.860 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-09-13T18:27:35.862 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-13T18:27:35.878 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.CsiU2qvJOQ 2024-09-13T18:27:35.878 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CsiU2qvJOQ 2024-09-13T18:27:35.946 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.CsiU2qvJOQ 2024-09-13T18:27:36.081 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.CsiU2qvJOQ is 6KB 2024-09-13T18:27:36.129 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.CsiU2qvJOQ 2024-09-13T18:27:36.144 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T18:27:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T18:27:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T18:27:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T18:27:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T18:27:36.214 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T18:27:36.369 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-09-13T18:28:06.395 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T18:28:06.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-13T18:28:06.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:29:06.468 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T18:29:15.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-13T18:29:15.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:29:16.062 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-13T18:29:16.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-13T18:29:16.500 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T18:29:16.500 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-09-13T18:29:16.515 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-505.el9.x86_64 2024-09-13T18:29:16.515 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T18:29:16.515 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:29:16.515 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T18:29:17.516 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T18:29:17.516 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:29:17.606 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2024-09-13T18:29:17.631 DEBUG:teuthology.parallel:result is None 2024-09-13T18:29:17.631 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T18:29:17.639 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T18:29:17.639 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:29:17.654 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T18:29:17.687 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T18:29:17.704 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T18:29:17.704 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:29:17.729 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T18:29:17.736 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T18:29:17.736 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:29:17.818 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:29:17.829 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:29:17.831 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T18:29:17.838 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T18:29:17.838 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:29:17.862 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T18:29:17.871 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T18:29:17.872 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:29:17.918 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:29:18.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:29:18.050 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:29:18.142 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:29:18.172 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:29:18.172 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:29:18.238 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-09-13T18:29:18.329 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:29:18.458 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T18:29:18.466 INFO:teuthology.task.internal:Starting timer... 2024-09-13T18:29:18.467 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T18:29:18.477 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T18:29:18.486 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T18:29:18.486 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-09-13T18:29:18.526 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-09-13T18:29:18.528 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T18:29:18.528 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-09-13T18:29:18.594 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-09-13T18:29:18.594 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2024-09-13T18:29:18.594 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T18:29:18.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:29:18.659 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T18:29:18.660 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-09-13T18:29:18.758 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T18:29:18.767 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T18:29:18.779 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T18:29:18.780 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T18:33:00.200 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-09-13T18:33:00.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-13T18:33:00.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:33:00.289 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-13T18:33:00.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-13T18:33:00.369 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T18:33:00.385 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T18:33:00.386 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:33:00.386 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:33:00.439 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T18:33:00.453 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T18:33:00.477 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-09-13T18:33:00.488 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-09-13T18:33:00.502 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T18:33:00.516 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T18:33:00.574 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:33:00.581 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:33:00.581 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-09-13T18:33:00.581 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:33:00.581 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:33:00.581 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:33:00.581 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:33:00.581 INFO:teuthology.run_tasks:Running task install... 2024-09-13T18:33:00.594 DEBUG:teuthology.task.install:project ceph 2024-09-13T18:33:00.594 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T18:33:00.594 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T18:33:00.594 INFO:teuthology.task.install:Using flavor: default 2024-09-13T18:33:00.599 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T18:33:00.599 INFO:teuthology.task.install:extra packages: ['librados2'] 2024-09-13T18:33:00.600 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T18:33:00.600 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:33:00.600 INFO:teuthology.packaging:ref: None 2024-09-13T18:33:00.600 INFO:teuthology.packaging:tag: None 2024-09-13T18:33:00.600 INFO:teuthology.packaging:branch: quincy 2024-09-13T18:33:00.600 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:33:00.601 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T18:33:00.840 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/eb16b97b08b1a378bbb6cbc2dadc21f07d8ac10b/centos/9/flavors/default/ 2024-09-13T18:33:00.841 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1629.geb16b97b 2024-09-13T18:33:00.964 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/eb16b97b08b1a378bbb6cbc2dadc21f07d8ac10b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/eb16b97b08b1a378bbb6cbc2dadc21f07d8ac10b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/eb16b97b08b1a378bbb6cbc2dadc21f07d8ac10b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T18:33:00.965 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:33:00.965 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T18:33:00.990 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-09-13T18:33:00.990 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:33:00.991 INFO:teuthology.packaging:ref: None 2024-09-13T18:33:00.991 INFO:teuthology.packaging:tag: None 2024-09-13T18:33:00.991 INFO:teuthology.packaging:branch: quincy 2024-09-13T18:33:00.991 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:33:00.991 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T18:33:01.056 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T18:33:01.131 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T18:33:01.196 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-09-13T18:33:01.197 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-09-13T18:33:01.425 INFO:teuthology.orchestra.run.smithi177.stdout:45 files removed 2024-09-13T18:33:01.452 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw 2024-09-13T18:33:02.000 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 291 kB/s | 77 kB 00:00 2024-09-13T18:33:02.284 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-13T18:33:02.560 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-13T18:33:03.218 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-13T18:33:06.553 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 20 MB 00:02 2024-09-13T18:33:12.936 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-13T18:33:14.753 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-13T18:33:15.791 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-13T18:33:21.272 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-13T18:33:23.614 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 14 M 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 3.1 M 2024-09-13T18:33:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 2.9 M 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 5.0 M 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 23 M 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 25 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 636 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 458 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T18:33:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 4.9 M 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 45 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 120 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 157 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 320 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 296 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 99 k 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:23.628 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:33:23.629 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:23.629 INFO:teuthology.orchestra.run.smithi177.stdout:Install 22 Packages 2024-09-13T18:33:23.629 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 2 Packages 2024-09-13T18:33:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:23.629 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 58 M 2024-09-13T18:33:23.629 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:33:24.214 INFO:teuthology.orchestra.run.smithi177.stdout:(1/24): ceph-base-17.2.7-1629.geb16b97b.el9.x86 12 MB/s | 5.0 MB 00:00 2024-09-13T18:33:24.281 INFO:teuthology.orchestra.run.smithi177.stdout:(2/24): ceph-selinux-17.2.7-1629.geb16b97b.el9. 378 kB/s | 25 kB 00:00 2024-09-13T18:33:24.382 INFO:teuthology.orchestra.run.smithi177.stdout:(3/24): libcephfs2-17.2.7-1629.geb16b97b.el9.x8 6.2 MB/s | 636 kB 00:00 2024-09-13T18:33:24.482 INFO:teuthology.orchestra.run.smithi177.stdout:(4/24): libradosstriper1-17.2.7-1629.geb16b97b. 4.5 MB/s | 458 kB 00:00 2024-09-13T18:33:24.807 INFO:teuthology.orchestra.run.smithi177.stdout:(5/24): librgw2-17.2.7-1629.geb16b97b.el9.x86_6 15 MB/s | 4.9 MB 00:00 2024-09-13T18:33:25.015 INFO:teuthology.orchestra.run.smithi177.stdout:(6/24): ceph-common-17.2.7-1629.geb16b97b.el9.x 18 MB/s | 23 MB 00:01 2024-09-13T18:33:25.032 INFO:teuthology.orchestra.run.smithi177.stdout:(7/24): python3-ceph-argparse-17.2.7-1629.geb16 200 kB/s | 45 kB 00:00 2024-09-13T18:33:25.058 INFO:teuthology.orchestra.run.smithi177.stdout:(8/24): python3-ceph-common-17.2.7-1629.geb16b9 2.8 MB/s | 120 kB 00:00 2024-09-13T18:33:25.083 INFO:teuthology.orchestra.run.smithi177.stdout:(9/24): python3-cephfs-17.2.7-1629.geb16b97b.el 3.0 MB/s | 157 kB 00:00 2024-09-13T18:33:25.108 INFO:teuthology.orchestra.run.smithi177.stdout:(10/24): python3-rados-17.2.7-1629.geb16b97b.el 6.2 MB/s | 320 kB 00:00 2024-09-13T18:33:25.134 INFO:teuthology.orchestra.run.smithi177.stdout:(11/24): python3-rbd-17.2.7-1629.geb16b97b.el9. 5.8 MB/s | 296 kB 00:00 2024-09-13T18:33:25.175 INFO:teuthology.orchestra.run.smithi177.stdout:(12/24): python3-rgw-17.2.7-1629.geb16b97b.el9. 1.5 MB/s | 99 kB 00:00 2024-09-13T18:33:25.317 INFO:teuthology.orchestra.run.smithi177.stdout:(13/24): ceph-radosgw-17.2.7-1629.geb16b97b.el9 9.4 MB/s | 14 MB 00:01 2024-09-13T18:33:25.384 INFO:teuthology.orchestra.run.smithi177.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2024-09-13T18:33:25.418 INFO:teuthology.orchestra.run.smithi177.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 564 kB/s | 160 kB 00:00 2024-09-13T18:33:25.460 INFO:teuthology.orchestra.run.smithi177.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-13T18:33:25.485 INFO:teuthology.orchestra.run.smithi177.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 630 kB/s | 42 kB 00:00 2024-09-13T18:33:25.536 INFO:teuthology.orchestra.run.smithi177.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-13T18:33:25.569 INFO:teuthology.orchestra.run.smithi177.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-09-13T18:33:25.594 INFO:teuthology.orchestra.run.smithi177.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-13T18:33:25.619 INFO:teuthology.orchestra.run.smithi177.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 822 kB/s | 67 kB 00:00 2024-09-13T18:33:25.686 INFO:teuthology.orchestra.run.smithi177.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-13T18:33:25.753 INFO:teuthology.orchestra.run.smithi177.stdout:(23/24): librados2-17.2.7-1629.geb16b97b.el9.x8 19 MB/s | 3.1 MB 00:00 2024-09-13T18:33:25.837 INFO:teuthology.orchestra.run.smithi177.stdout:(24/24): librbd1-17.2.7-1629.geb16b97b.el9.x86_ 14 MB/s | 2.9 MB 00:00 2024-09-13T18:33:25.842 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:33:25.842 INFO:teuthology.orchestra.run.smithi177.stdout:Total 26 MB/s | 58 MB 00:02 2024-09-13T18:33:25.904 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:33:25.937 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:33:25.937 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:33:26.617 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:33:26.617 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:33:27.881 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:33:28.246 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-13T18:33:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 2/26 2024-09-13T18:33:28.417 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 2/26 2024-09-13T18:33:28.503 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:17.2.7-1629.geb16b97b.el9.x86_64 3/26 2024-09-13T18:33:28.526 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 4/26 2024-09-13T18:33:28.598 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 4/26 2024-09-13T18:33:28.644 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-13T18:33:28.673 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-13T18:33:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-13T18:33:29.145 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 8/26 2024-09-13T18:33:29.208 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 8/26 2024-09-13T18:33:29.245 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-13T18:33:29.288 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:17.2.7-1629.geb16b97b.el9. 10/26 2024-09-13T18:33:29.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:17.2.7-1629.geb16b97b.el9.x86_64 11/26 2024-09-13T18:33:29.386 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:17.2.7-1629.geb16b97b.el9.x86_64 12/26 2024-09-13T18:33:29.414 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_6 13/26 2024-09-13T18:33:29.480 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_6 13/26 2024-09-13T18:33:29.524 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-13T18:33:29.556 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-13T18:33:29.589 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-13T18:33:29.882 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-13T18:33:29.935 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 18/26 2024-09-13T18:33:30.025 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 18/26 2024-09-13T18:33:30.099 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:17.2.7-1629.geb16b97b.el9.x86_64 19/26 2024-09-13T18:33:30.137 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:17.2.7-1629.geb16b97b.el9.x8 20/26 2024-09-13T18:33:32.557 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 21/26 2024-09-13T18:33:32.625 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 21/26 2024-09-13T18:33:33.146 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 21/26 2024-09-13T18:33:33.187 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 22/26 2024-09-13T18:33:33.282 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 22/26 2024-09-13T18:33:33.282 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T18:33:33.282 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T18:33:33.282 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:33.313 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 23/26 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 23/26 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-09-13T18:33:43.219 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:44.547 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 24/26 2024-09-13T18:33:44.625 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 24/26 2024-09-13T18:33:44.625 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:33:44.625 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T18:33:44.625 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:33:44.625 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:33:44.625 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:44.626 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-13T18:33:44.707 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-13T18:33:44.708 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-13T18:33:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-13T18:33:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 1/26 2024-09-13T18:33:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 2/26 2024-09-13T18:33:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 3/26 2024-09-13T18:33:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 4/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 5/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_6 6/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 7/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1629.geb16b97b.el9. 8/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:17.2.7-1629.geb16b97b.el9.x8 9/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:17.2.7-1629.geb16b97b.el9.x86_64 10/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:17.2.7-1629.geb16b97b.el9.x86_64 11/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:17.2.7-1629.geb16b97b.el9.x86_64 12/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:17.2.7-1629.geb16b97b.el9.x86_64 13/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-13T18:33:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 23/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-13T18:33:45.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 25/26 2024-09-13T18:33:46.999 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T18:33:47.000 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.001 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T18:33:47.002 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.002 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.002 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:47.002 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T18:33:47.002 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:47.002 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:33:47.333 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-test 2024-09-13T18:33:47.924 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:26 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:33:48.055 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 48 M 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 163 k 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T18:33:48.056 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout:Install 4 Packages 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 48 M 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 193 M 2024-09-13T18:33:48.057 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:33:48.413 INFO:teuthology.orchestra.run.smithi177.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-13T18:33:48.438 INFO:teuthology.orchestra.run.smithi177.stdout:(2/4): libcephsqlite-17.2.7-1629.geb16b97b.el9. 652 kB/s | 163 kB 00:00 2024-09-13T18:33:48.454 INFO:teuthology.orchestra.run.smithi177.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-13T18:33:49.671 INFO:teuthology.orchestra.run.smithi177.stdout:(4/4): ceph-test-17.2.7-1629.geb16b97b.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-13T18:33:49.673 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:33:49.674 INFO:teuthology.orchestra.run.smithi177.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-13T18:33:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:33:49.721 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:33:49.721 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:33:50.085 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:33:50.086 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:33:50.623 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:33:50.730 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T18:33:50.774 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T18:33:50.798 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 3/4 2024-09-13T18:33:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 3/4 2024-09-13T18:33:55.271 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:17.2.7-1629.geb16b97b.el9.x86_64 4/4 2024-09-13T18:33:57.469 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-test-2:17.2.7-1629.geb16b97b.el9.x86_64 4/4 2024-09-13T18:33:57.469 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:17.2.7-1629.geb16b97b.el9.x86_64 1/4 2024-09-13T18:33:57.469 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 2/4 2024-09-13T18:33:57.469 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T18:33:57.826 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:57.827 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:33:58.082 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph 2024-09-13T18:33:58.651 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:37 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:33:58.789 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:33:58.791 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 6.5 k 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 2.1 M 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 1.4 M 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1629.geb16b97b.el9 ceph-noarch 240 k 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 3.9 M 2024-09-13T18:33:58.792 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 15 M 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T18:33:58.793 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T18:33:58.794 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout:Install 34 Packages 2024-09-13T18:33:58.795 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:33:58.796 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 27 M 2024-09-13T18:33:58.796 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 96 M 2024-09-13T18:33:58.796 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:33:59.005 INFO:teuthology.orchestra.run.smithi177.stdout:(1/34): ceph-17.2.7-1629.geb16b97b.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-13T18:33:59.205 INFO:teuthology.orchestra.run.smithi177.stdout:(2/34): ceph-mds-17.2.7-1629.geb16b97b.el9.x86_ 6.5 MB/s | 2.1 MB 00:00 2024-09-13T18:33:59.247 INFO:teuthology.orchestra.run.smithi177.stdout:(3/34): ceph-mgr-17.2.7-1629.geb16b97b.el9.x86_ 3.9 MB/s | 1.4 MB 00:00 2024-09-13T18:33:59.339 INFO:teuthology.orchestra.run.smithi177.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1629.geb16 2.6 MB/s | 240 kB 00:00 2024-09-13T18:33:59.522 INFO:teuthology.orchestra.run.smithi177.stdout:(5/34): ceph-mon-17.2.7-1629.geb16b97b.el9.x86_ 7.6 MB/s | 3.9 MB 00:00 2024-09-13T18:33:59.706 INFO:teuthology.orchestra.run.smithi177.stdout:(6/34): ceph-osd-17.2.7-1629.geb16b97b.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-13T18:34:00.324 INFO:teuthology.orchestra.run.smithi177.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 257 kB/s | 253 kB 00:00 2024-09-13T18:34:00.440 INFO:teuthology.orchestra.run.smithi177.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 146 kB/s | 106 kB 00:00 2024-09-13T18:34:00.591 INFO:teuthology.orchestra.run.smithi177.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-13T18:34:01.066 INFO:teuthology.orchestra.run.smithi177.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 182 kB/s | 135 kB 00:00 2024-09-13T18:34:01.133 INFO:teuthology.orchestra.run.smithi177.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 181 kB/s | 125 kB 00:00 2024-09-13T18:34:01.191 INFO:teuthology.orchestra.run.smithi177.stdout:(12/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-13T18:34:01.217 INFO:teuthology.orchestra.run.smithi177.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 416 kB/s | 35 kB 00:00 2024-09-13T18:34:01.242 INFO:teuthology.orchestra.run.smithi177.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-13T18:34:01.267 INFO:teuthology.orchestra.run.smithi177.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-13T18:34:01.292 INFO:teuthology.orchestra.run.smithi177.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-13T18:34:01.343 INFO:teuthology.orchestra.run.smithi177.stdout:(17/34): python3-urllib3-1.26.5-6.el9.noarch.rp 286 kB/s | 215 kB 00:00 2024-09-13T18:34:01.368 INFO:teuthology.orchestra.run.smithi177.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-13T18:34:01.401 INFO:teuthology.orchestra.run.smithi177.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 405 kB/s | 11 kB 00:00 2024-09-13T18:34:01.435 INFO:teuthology.orchestra.run.smithi177.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-13T18:34:01.460 INFO:teuthology.orchestra.run.smithi177.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 310 kB/s | 18 kB 00:00 2024-09-13T18:34:01.485 INFO:teuthology.orchestra.run.smithi177.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-13T18:34:01.511 INFO:teuthology.orchestra.run.smithi177.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-13T18:34:01.536 INFO:teuthology.orchestra.run.smithi177.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-13T18:34:01.561 INFO:teuthology.orchestra.run.smithi177.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-13T18:34:01.587 INFO:teuthology.orchestra.run.smithi177.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T18:34:01.621 INFO:teuthology.orchestra.run.smithi177.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-13T18:34:01.646 INFO:teuthology.orchestra.run.smithi177.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-09-13T18:34:01.671 INFO:teuthology.orchestra.run.smithi177.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-13T18:34:01.696 INFO:teuthology.orchestra.run.smithi177.stdout:(30/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-13T18:34:01.722 INFO:teuthology.orchestra.run.smithi177.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 473 kB/s | 36 kB 00:00 2024-09-13T18:34:01.747 INFO:teuthology.orchestra.run.smithi177.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 807 kB/s | 20 kB 00:00 2024-09-13T18:34:01.772 INFO:teuthology.orchestra.run.smithi177.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-13T18:34:01.805 INFO:teuthology.orchestra.run.smithi177.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-13T18:34:01.810 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:01.810 INFO:teuthology.orchestra.run.smithi177.stdout:Total 9.0 MB/s | 27 MB 00:03 2024-09-13T18:34:01.935 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:01.972 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:01.972 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:02.365 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:02.366 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:03.628 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:03.678 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-13T18:34:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-13T18:34:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-13T18:34:03.844 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-13T18:34:03.922 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-13T18:34:03.963 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-13T18:34:03.998 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-13T18:34:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-13T18:34:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-13T18:34:04.096 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-13T18:34:04.136 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-13T18:34:04.171 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-13T18:34:04.206 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-13T18:34:04.254 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-13T18:34:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-13T18:34:04.368 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-13T18:34:04.507 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-13T18:34:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-13T18:34:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-13T18:34:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-13T18:34:04.743 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-13T18:34:04.798 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-13T18:34:04.984 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-13T18:34:05.040 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-13T18:34:05.125 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-13T18:34:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-13T18:34:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-13T18:34:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-13T18:34:05.539 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1629.geb16b97b.el9. 29/34 2024-09-13T18:34:05.574 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 30/34 2024-09-13T18:34:05.601 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 30/34 2024-09-13T18:34:05.601 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:34:05.601 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T18:34:05.601 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:34:05.601 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:34:05.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:06.909 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 31/34 2024-09-13T18:34:06.936 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 31/34 2024-09-13T18:34:06.936 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:34:06.936 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T18:34:06.936 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:34:06.936 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:34:06.936 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:07.317 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 32/34 2024-09-13T18:34:07.341 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 32/34 2024-09-13T18:34:07.342 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:34:07.342 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T18:34:07.342 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:34:07.342 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:34:07.342 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:07.543 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 33/34 2024-09-13T18:34:07.567 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 33/34 2024-09-13T18:34:07.567 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:34:07.567 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T18:34:07.568 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:34:07.568 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:34:07.568 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:07.632 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:17.2.7-1629.geb16b97b.el9.x86_64 34/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-2:17.2.7-1629.geb16b97b.el9.x86_64 34/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:17.2.7-1629.geb16b97b.el9.x86_64 1/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 2/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 3/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 4/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 5/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1629.geb16b97b.el9. 6/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-13T18:34:09.262 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-13T18:34:09.263 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-13T18:34:09.264 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-13T18:34:09.265 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-13T18:34:09.265 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-13T18:34:09.265 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-13T18:34:09.265 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-13T18:34:10.174 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-13T18:34:10.174 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:10.174 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:10.174 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:10.174 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:17.2.7-1629.geb16b97b.el9.noarch 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T18:34:10.175 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T18:34:10.176 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T18:34:10.177 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T18:34:10.178 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T18:34:10.178 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T18:34:10.178 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T18:34:10.178 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T18:34:10.178 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:10.178 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:10.539 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-base 2024-09-13T18:34:11.136 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:11.226 INFO:teuthology.orchestra.run.smithi177.stdout:Package ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:11.267 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:11.268 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:11.268 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:11.338 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T18:34:11.912 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 137 k 2024-09-13T18:34:12.044 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:12.045 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:12.045 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:12.045 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-09-13T18:34:12.045 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:12.045 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 137 k 2024-09-13T18:34:12.046 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 379 k 2024-09-13T18:34:12.046 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:12.259 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-immutable-object-cache-17.2.7-1629.geb16b9 646 kB/s | 137 kB 00:00 2024-09-13T18:34:12.259 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:12.259 INFO:teuthology.orchestra.run.smithi177.stdout:Total 641 kB/s | 137 kB 00:00 2024-09-13T18:34:12.259 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:12.269 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:12.269 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:12.324 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:12.325 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:12.491 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:12.543 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1629.geb16b97b. 1/1 2024-09-13T18:34:12.568 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1629.geb16b97b. 1/1 2024-09-13T18:34:12.568 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:34:12.568 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T18:34:12.568 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:13.532 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1629.geb16b97b. 1/1 2024-09-13T18:34:13.532 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:13.532 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:13.532 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:13.532 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:13.532 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:13.761 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi177.stdout:Package ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:14.546 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-fuse 2024-09-13T18:34:15.114 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:15.248 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:15.248 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:15.248 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-13T18:34:15.248 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:15.248 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 813 k 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout:Install 2 Packages 2024-09-13T18:34:15.249 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:15.250 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 893 k 2024-09-13T18:34:15.250 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 2.6 M 2024-09-13T18:34:15.250 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:15.545 INFO:teuthology.orchestra.run.smithi177.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 530 kB/s | 80 kB 00:00 2024-09-13T18:34:15.687 INFO:teuthology.orchestra.run.smithi177.stdout:(2/2): ceph-fuse-17.2.7-1629.geb16b97b.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-13T18:34:15.688 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:15.688 INFO:teuthology.orchestra.run.smithi177.stdout:Total 2.0 MB/s | 893 kB 00:00 2024-09-13T18:34:15.700 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:15.722 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:15.722 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:15.796 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:15.796 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:15.990 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:16.124 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T18:34:16.192 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 2/2 2024-09-13T18:34:16.916 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 2/2 2024-09-13T18:34:16.916 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 1/2 2024-09-13T18:34:17.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T18:34:17.219 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:17.219 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:17.219 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T18:34:17.219 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:17.219 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:17.448 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados-devel 2024-09-13T18:34:18.031 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:57 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:18.162 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:18.162 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:18.162 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-09-13T18:34:18.162 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:18.162 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 126 k 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 126 k 2024-09-13T18:34:18.163 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 448 k 2024-09-13T18:34:18.164 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:18.350 INFO:teuthology.orchestra.run.smithi177.stdout:librados-devel-17.2.7-1629.geb16b97b.el9.x86_64 679 kB/s | 126 kB 00:00 2024-09-13T18:34:18.350 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:18.351 INFO:teuthology.orchestra.run.smithi177.stdout:Total 674 kB/s | 126 kB 00:00 2024-09-13T18:34:18.351 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:18.356 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:18.356 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:18.413 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:18.413 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:18.563 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:19.051 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:19.294 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:19.294 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:19.295 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:19.295 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:19.295 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:19.295 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:19.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install libcephfs2 2024-09-13T18:34:20.083 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:20.175 INFO:teuthology.orchestra.run.smithi177.stdout:Package libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:20.219 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:20.220 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:20.220 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:20.305 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install libcephfs-devel 2024-09-13T18:34:20.867 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:20.997 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:20.997 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 25 k 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 25 k 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 110 k 2024-09-13T18:34:20.998 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:21.145 INFO:teuthology.orchestra.run.smithi177.stdout:libcephfs-devel-17.2.7-1629.geb16b97b.el9.x86_6 170 kB/s | 25 kB 00:00 2024-09-13T18:34:21.146 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:21.146 INFO:teuthology.orchestra.run.smithi177.stdout:Total 169 kB/s | 25 kB 00:00 2024-09-13T18:34:21.146 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:21.149 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:21.149 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:21.151 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:21.151 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:21.230 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:21.340 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:21.700 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:22.126 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados2 2024-09-13T18:34:22.682 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:22.771 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:22.813 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:22.814 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:22.814 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:22.885 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librbd1 2024-09-13T18:34:23.442 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:02 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:23.532 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:23.573 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:23.574 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:23.574 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:23.644 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rados 2024-09-13T18:34:24.200 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:03 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:24.290 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rados-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:24.331 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:24.332 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:24.332 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:24.405 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rgw 2024-09-13T18:34:24.963 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:03 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:25.054 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rgw-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:25.095 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:25.096 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:25.096 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:25.172 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-cephfs 2024-09-13T18:34:25.727 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:25.817 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-cephfs-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:25.858 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:25.859 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:25.859 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:25.943 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rbd 2024-09-13T18:34:26.497 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:26.587 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rbd-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:26.628 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:26.629 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:26.630 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:26.702 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-fuse 2024-09-13T18:34:27.257 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:27.389 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:27.390 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:27.390 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-09-13T18:34:27.390 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 82 k 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 82 k 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 222 k 2024-09-13T18:34:27.391 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:27.589 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-fuse-17.2.7-1629.geb16b97b.el9.x86_64.rpm 413 kB/s | 82 kB 00:00 2024-09-13T18:34:27.590 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:27.590 INFO:teuthology.orchestra.run.smithi177.stdout:Total 410 kB/s | 82 kB 00:00 2024-09-13T18:34:27.590 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:27.597 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:27.597 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:27.654 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:27.654 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:27.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:28.244 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:28.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:28.528 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:28.529 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:28.529 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:28.529 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:28.529 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:28.744 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-mirror 2024-09-13T18:34:29.309 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:29.446 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 2.9 M 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 2.9 M 2024-09-13T18:34:29.447 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 12 M 2024-09-13T18:34:29.448 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:29.794 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-mirror-17.2.7-1629.geb16b97b.el9.x86_64.rpm 8.5 MB/s | 2.9 MB 00:00 2024-09-13T18:34:29.795 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:29.795 INFO:teuthology.orchestra.run.smithi177.stdout:Total 8.4 MB/s | 2.9 MB 00:00 2024-09-13T18:34:29.795 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:29.805 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:29.805 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:29.878 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:29.878 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:30.283 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:30.345 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:30.370 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:30.370 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:34:30.371 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T18:34:30.371 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:34:30.371 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:34:30.371 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:31.478 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:31.478 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:31.714 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-nbd 2024-09-13T18:34:32.285 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:32.426 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:17.2.7-1629.geb16b97b.el9 ceph 165 k 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 165 k 2024-09-13T18:34:32.427 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 477 k 2024-09-13T18:34:32.428 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:34:32.640 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-nbd-17.2.7-1629.geb16b97b.el9.x86_64.rpm 780 kB/s | 165 kB 00:00 2024-09-13T18:34:32.640 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:34:32.641 INFO:teuthology.orchestra.run.smithi177.stdout:Total 772 kB/s | 165 kB 00:00 2024-09-13T18:34:32.641 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:34:32.649 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:34:32.649 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:34:32.709 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:34:32.709 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:34:32.861 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:34:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:33.407 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:33.628 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:17.2.7-1629.geb16b97b.el9.x86_64 1/1 2024-09-13T18:34:33.628 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:33.628 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:34:33.628 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:17.2.7-1629.geb16b97b.el9.x86_64 2024-09-13T18:34:33.628 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:33.628 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:33.891 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados2 2024-09-13T18:34:34.463 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Sep 2024 06:33:21 PM UTC. 2024-09-13T18:34:34.556 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 is already installed. 2024-09-13T18:34:34.599 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:34:34.599 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-13T18:34:34.600 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:34:34.674 DEBUG:teuthology.parallel:result is None 2024-09-13T18:34:34.675 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:34:34.675 INFO:teuthology.packaging:ref: None 2024-09-13T18:34:34.675 INFO:teuthology.packaging:tag: None 2024-09-13T18:34:34.675 INFO:teuthology.packaging:branch: quincy 2024-09-13T18:34:34.675 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:34:34.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T18:34:34.938 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T18:34:34.959 INFO:teuthology.orchestra.run.smithi177.stdout:17.2.7-1629.geb16b97b.el9 2024-09-13T18:34:34.960 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1629.geb16b97b.el9 2024-09-13T18:34:34.960 INFO:teuthology.task.install:The correct ceph version 17.2.7-1629.geb16b97b is installed. 2024-09-13T18:34:34.962 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T18:34:34.963 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:34.963 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:34:35.024 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T18:34:35.024 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:35.025 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:34:35.089 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:34:35.154 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T18:34:35.154 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:35.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:34:35.219 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:34:35.283 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T18:34:35.284 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:35.284 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:34:35.348 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:34:35.416 INFO:teuthology.run_tasks:Running task print... 2024-09-13T18:34:35.425 INFO:teuthology.task.print:**** done installing quincy 2024-09-13T18:34:35.425 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T18:34:35.512 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T18:34:35.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2024-09-13T18:34:35.537 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T18:34:35.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T18:34:35.601 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T18:34:35.602 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T18:34:35.675 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T18:34:35.675 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T18:34:35.675 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}}', 'email': None, 'first_in_suite': False, 'job_id': '7903993', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/quincy latest\n'}], 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'targets': {'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgtfPDrkS1U+OTKhEY3eqaaqlHpNjaFiq6uTr9JoH/BGmg6EX8v6uBA5+mVe5LIHo+u0H85+J6qv9fyCu17J+c='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'quincy', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'print': '**** done installing quincy'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client quincy']}}, {'print': '**** done ceph'}, {'print': '*** upgrading, no cephfs present'}, {'exec': {'mon.a': ['ceph fs dump']}}, {'install.upgrade': {'mon.a': None}}, {'print': '**** done install.upgrade'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph versions', 'ceph osd dump -f json-pretty', 'ceph fs dump', 'ceph osd require-osd-release reef', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T18:34:35.676 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T18:34:35.723 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T18:34:35.787 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:35.788 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:34:35.843 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:34:35.844 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-13 18:34:30.638102108 +0000 2024-09-13T18:34:35.899 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-13 18:32:41.426187675 +0000 2024-09-13T18:34:35.900 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-13 18:32:41.426187675 +0000 2024-09-13T18:34:35.900 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-13 18:32:41.426187675 +0000 2024-09-13T18:34:35.900 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:34:35.962 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-13T18:34:35.962 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-13T18:34:35.962 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000205827 s, 2.5 MB/s 2024-09-13T18:34:35.963 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:34:36.020 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-09-13T18:34:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-13 18:34:30.638102108 +0000 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-13 18:32:41.757178043 +0000 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-13 18:32:41.757178043 +0000 2024-09-13T18:34:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-13 18:32:41.757178043 +0000 2024-09-13T18:34:36.076 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:34:36.139 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-13T18:34:36.139 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-13T18:34:36.139 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000187606 s, 2.7 MB/s 2024-09-13T18:34:36.140 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:34:36.197 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-09-13T18:34:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:34:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:34:36.252 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T18:34:36.252 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:34:36.252 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:34:36.252 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-13 18:34:30.639102081 +0000 2024-09-13T18:34:36.253 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-13 18:32:42.071168905 +0000 2024-09-13T18:34:36.253 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-13 18:32:42.071168905 +0000 2024-09-13T18:34:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-13 18:32:42.071168905 +0000 2024-09-13T18:34:36.253 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:34:36.316 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-13T18:34:36.316 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-13T18:34:36.316 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000209104 s, 2.4 MB/s 2024-09-13T18:34:36.317 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:34:36.374 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-09-13T18:34:36.429 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:34:36.429 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:34:36.429 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-13T18:34:36.429 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:34:36.429 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:34:36.429 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-13 18:34:30.639102081 +0000 2024-09-13T18:34:36.430 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-13 18:32:42.409159069 +0000 2024-09-13T18:34:36.430 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-13 18:32:42.409159069 +0000 2024-09-13T18:34:36.430 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-13 18:32:42.409159069 +0000 2024-09-13T18:34:36.430 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:34:36.492 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-13T18:34:36.493 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-13T18:34:36.493 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000225303 s, 2.3 MB/s 2024-09-13T18:34:36.494 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:34:36.549 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:34:36.550 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-13T18:34:36.550 INFO:tasks.ceph:Generating config... 2024-09-13T18:34:36.551 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T18:34:36.551 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-13T18:34:36.551 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2024-09-13T18:34:36.551 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2024-09-13T18:34:36.551 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-09-13T18:34:36.552 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T18:34:36.552 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T18:34:36.552 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T18:34:36.552 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T18:34:36.552 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T18:34:36.552 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2024-09-13T18:34:36.553 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T18:34:36.553 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T18:34:36.553 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T18:34:36.553 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T18:34:36.553 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T18:34:36.554 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T18:34:36.555 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T18:34:36.555 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T18:34:36.666 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T18:34:36.669 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T18:34:36.708 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:34:36.773 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.177'), ('mon.b', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]'), ('mon.c', '[v2:172.21.15.177:3302,v1:172.21.15.177:6791]')] 2024-09-13T18:34:36.773 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790],[v2:172.21.15.177:3302,v1:172.21.15.177:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T18:34:36.774 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:36.774 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T18:34:36.828 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.177 --addv b '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --addv c '[v2:172.21.15.177:3302,v1:172.21.15.177:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:36.910 INFO:teuthology.orchestra.run.smithi177.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:monmaptool: generated fsid af843e4a-307e-4c3a-83b8-4495d5352147 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:setting min_mon_release = octopus 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:epoch 0 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:fsid af843e4a-307e-4c3a-83b8-4495d5352147 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:last_changed 2024-09-13T18:34:36.909253+0000 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:created 2024-09-13T18:34:36.909253+0000 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:min_mon_release 15 (octopus) 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:election_strategy: 1 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:0: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.a 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:1: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.b 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:2: [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] mon.c 2024-09-13T18:34:36.911 INFO:teuthology.orchestra.run.smithi177.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T18:34:36.912 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T18:34:36.968 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID af843e4a-307e-4c3a-83b8-4495d5352147... 2024-09-13T18:34:36.968 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-09-13T18:34:37.051 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-09-13T18:34:37.052 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.053 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790],[v2:172.21.15.177:3302,v1:172.21.15.177:6791]" 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on pool no app = False 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: ms bind msgr2 = False 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore warn on legacy statfs = False 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore warn on no per pool omap = False 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn min per osd = 0 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = af843e4a-307e-4c3a-83b8-4495d5352147 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-09-13T18:34:37.054 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-09-13T18:34:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2024-09-13T18:34:37.056 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-09-13T18:34:37.057 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-13T18:34:37.058 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:34:37.059 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = False 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on pool no app = False 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-09-13T18:34:37.060 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2024-09-13T18:34:37.061 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-09-13T18:34:37.062 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-09-13T18:34:37.063 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T18:34:37.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T18:34:37.153 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T18:34:37.153 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:37.153 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T18:34:37.170 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:37.170 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T18:34:37.225 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:34:37.225 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:37.225 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T18:34:37.225 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:34:37.300 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:37.300 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:37.355 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T18:34:37.355 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T18:34:37.355 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T18:34:37.443 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T18:34:37.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T18:34:37.491 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T18:34:37.494 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T18:34:37.494 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T18:34:37.494 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T18:34:37.494 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-13T18:34:37.494 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T18:34:37.519 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:34:37.519 INFO:tasks.ceph:role: osd.0 2024-09-13T18:34:37.519 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:34:37.519 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T18:34:37.591 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:34:37.592 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:34:38.105 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-13T18:34:38.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-13T18:34:38.107 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T18:34:38.212 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T18:34:38.242 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T18:34:38.307 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:34:38.307 INFO:tasks.ceph:role: osd.1 2024-09-13T18:34:38.308 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:34:38.308 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T18:34:38.380 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:34:38.380 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:34:38.380 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:34:38.380 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T18:34:38.381 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:34:38.381 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:34:38.381 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:34:38.381 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T18:34:38.381 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:34:38.381 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:34:38.839 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-13T18:34:38.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-13T18:34:38.841 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T18:34:38.934 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T18:34:38.963 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T18:34:39.030 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:34:39.030 INFO:tasks.ceph:role: osd.2 2024-09-13T18:34:39.030 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:34:39.031 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T18:34:39.102 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:34:39.102 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:34:39.102 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:34:39.103 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-13T18:34:39.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-13T18:34:39.602 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T18:34:39.721 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T18:34:39.750 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T18:34:39.816 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:34:39.816 INFO:tasks.ceph:role: osd.3 2024-09-13T18:34:39.816 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:34:39.817 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T18:34:39.888 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:34:39.888 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:34:39.888 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:34:39.888 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T18:34:39.888 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:34:39.888 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:34:39.889 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:34:39.889 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T18:34:39.889 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:34:39.889 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:34:40.400 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-13T18:34:40.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-13T18:34:40.407 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T18:34:40.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T18:34:40.569 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:40.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:40.667+0000 7fdf28525440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T18:34:40.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:40.668+0000 7fdf28525440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T18:34:40.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:40.668+0000 7fdf28525440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:34:40.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:40.668+0000 7fdf28525440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:34:40.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:40.668+0000 7fdf28525440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:34:40.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:40.668+0000 7fdf28525440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T18:34:41.201 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:41.198+0000 7fdf28525440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:41.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:41.973+0000 7fdf28525440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:42.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:42.739+0000 7fdf28525440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:43.543 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T18:34:43.572 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:43.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:43.655+0000 7f40a5f49440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T18:34:43.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:43.655+0000 7f40a5f49440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T18:34:43.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:43.655+0000 7f40a5f49440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:34:43.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:43.655+0000 7f40a5f49440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:34:43.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:43.655+0000 7f40a5f49440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:34:43.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:43.656+0000 7f40a5f49440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T18:34:44.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:44.181+0000 7f40a5f49440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:44.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:44.943+0000 7f40a5f49440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:45.709 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:45.706+0000 7f40a5f49440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:46.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T18:34:46.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:46.614+0000 7f1872430440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T18:34:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:46.615+0000 7f1872430440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T18:34:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:46.615+0000 7f1872430440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:34:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:46.615+0000 7f1872430440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:34:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:46.615+0000 7f1872430440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:34:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:46.615+0000 7f1872430440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T18:34:47.149 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:47.146+0000 7f1872430440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:47.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:47.910+0000 7f1872430440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:48.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:48.675+0000 7f1872430440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:49.480 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T18:34:49.506 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:49.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:49.585+0000 7fb72971d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T18:34:49.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:49.585+0000 7fb72971d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T18:34:49.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:49.585+0000 7fb72971d440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:34:49.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:49.585+0000 7fb72971d440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:34:49.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:49.585+0000 7fb72971d440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:34:49.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:49.586+0000 7fb72971d440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T18:34:50.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:50.108+0000 7fb72971d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:50.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:50.876+0000 7fb72971d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:51.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:51.642+0000 7fb72971d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T18:34:52.442 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T18:34:52.512 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T18:34:52.512 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:52.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T18:34:52.581 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:52.581 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T18:34:52.649 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:52.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T18:34:52.715 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:52.716 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T18:34:52.782 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:52.782 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T18:34:52.847 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:52.847 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T18:34:52.912 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T18:34:52.912 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T18:34:52.975 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2024-09-13T18:34:52.975 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQC9heRm01ZXGhAAh8Wk7ZYNzpB0vND0DzJCUw== 2024-09-13T18:34:52.975 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQC9heRmDdw8HRAAeaoZaSOc/wsgcKZqihXG0Q== 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQDAheRmlCLmJxAArySxx6GT/p33sMXNg53l5g== 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQDDheRmS7knJxAAwj6Xp469nP2QWKksJKcfSg== 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQDGheRmYXe7JBAAc/EWnHwKJVkRNmmnQQiWIQ== 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2024-09-13T18:34:52.976 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQDJheRmWjn8IhAAQrRNnzcLhTfBJ3MbKvOfVA== 2024-09-13T18:34:52.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:34:53.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:34:53.110 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:34:53.156 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:34:53.202 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:34:53.248 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:34:53.295 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T18:34:53.295 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T18:34:53.319 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:34:53.411 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T18:34:53.436 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T18:34:53.500 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:34:53.592 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T18:34:53.616 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T18:34:53.681 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:34:53.772 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T18:34:53.797 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:34:53.855 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T18:34:53.855 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:34:53.855 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:34:53.897 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:34:53.897 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:34:53.897 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:34:53.899 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:34:53.899 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:34:53.899 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:34:53.901 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:34:53.901 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T18:34:53.902 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T18:34:53.902 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T18:34:53.903 INFO:tasks.ceph.mgr.x:Started 2024-09-13T18:34:53.904 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T18:34:53.904 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T18:34:53.905 INFO:tasks.ceph.mgr.y:Started 2024-09-13T18:34:53.906 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T18:34:53.906 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T18:34:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.028+0000 7fddcad9d640 1 Processor -- start 2024-09-13T18:34:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.028+0000 7fddcad9d640 1 -- start start 2024-09-13T18:34:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddcad9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 0x7fddc412f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddcad9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddc4131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddcad9d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40937a0 con 0x7fddc4068260 2024-09-13T18:34:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddcad9d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc406a020 con 0x7fddc4068d40 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddcad9d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc406a250 con 0x7fddc412f9e0 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc3fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 0x7fddc412f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc8b39640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fddc4068260 0x7fddc4068630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:50628/0 (socket says 172.21.15.177:50628) 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc933a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddc4131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc8b39640 1 -- 172.21.15.177:0/1178520988 learned_addr learned my addr 172.21.15.177:0/1178520988 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc933a640 1 -- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 msgr2=0x7fddc4131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc933a640 1 -- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 msgr2=0x7fddc4131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc933a640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddc4131dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc933a640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddc4131dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc3fff640 1 -- 172.21.15.177:0/1178520988 >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 msgr2=0x7fddc412f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc3fff640 1 -- 172.21.15.177:0/1178520988 >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 msgr2=0x7fddc412f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:34:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc3fff640 1 --2- 172.21.15.177:0/1178520988 >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 0x7fddc412f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T18:34:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.029+0000 7fddc3fff640 1 --2- 172.21.15.177:0/1178520988 >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 0x7fddc412f4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271358081 0 0) 0x7fddc40937a0 con 0x7fddc4068260 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb40034e0 con 0x7fddc4068260 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 <== mon.0 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019874486 0 0) 0x7fddb40034e0 con 0x7fddc4068260 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 msgr2=0x7fddc4131dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddc4131dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 msgr2=0x7fddc412f4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:34:54.226 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 --2- 172.21.15.177:0/1178520988 >> v2:172.21.15.177:3300/0 conn(0x7fddc4068d40 0x7fddc412f4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40699f0 con 0x7fddc4068260 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 <== mon.0 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3709956878 0 0) 0x7fddb8001470 con 0x7fddc4068260 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 >> v1:172.21.15.177:6789/0 conn(0x7fddc4068260 legacy=0x7fddc4068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddb4003a50 0x7fddb4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddb4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddb4004cc0 0x7fddb4005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc406a020 con 0x7fddb4003a50 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc406a250 con 0x7fddb4004cc0 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400bc60 con 0x7fddc412f9e0 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc8b39640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddb4003a50 0x7fddb4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc3fff640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddb4004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc933a640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddb4004cc0 0x7fddb4005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc3fff640 1 -- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddb4004cc0 msgr2=0x7fddb4005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:54.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc3fff640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddb4004cc0 0x7fddb4005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc3fff640 1 -- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddb4003a50 msgr2=0x7fddb4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:54.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc3fff640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddb4003a50 0x7fddb4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.223+0000 7fddc3fff640 1 -- 172.21.15.177:0/1178520988 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40034e0 con 0x7fddc412f9e0 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.224+0000 7fddc3fff640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddb4004780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006fd0 tx=0x7fddb0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bbe4776e3a8f95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.224+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb002abd0 con 0x7fddc412f9e0 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.224+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb002e070 con 0x7fddc412f9e0 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddc37fe640 1 -- 172.21.15.177:0/1178520988 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0034410 con 0x7fddc412f9e0 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddcad9d640 1 -- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 msgr2=0x7fddb4004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddcad9d640 1 --2- 172.21.15.177:0/1178520988 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc412f9e0 0x7fddb4004780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006fd0 tx=0x7fddb0008040 comp rx=0 tx=0).stop 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddc933a640 1 -- 172.21.15.177:0/1178520988 reap_dead start 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddcad9d640 1 -- 172.21.15.177:0/1178520988 shutdown_connections 2024-09-13T18:34:54.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddcad9d640 1 -- 172.21.15.177:0/1178520988 >> 172.21.15.177:0/1178520988 conn(0x7fddc4099e10 msgr2=0x7fddc409a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddcad9d640 1 -- 172.21.15.177:0/1178520988 shutdown_connections 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.225+0000 7fddcad9d640 1 -- 172.21.15.177:0/1178520988 wait complete. 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.226+0000 7fddcad9d640 1 Processor -- start 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.226+0000 7fddcad9d640 1 -- start start 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddcad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 0x7fddc412f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddcad9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc408db50 0x7fddc408df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddcad9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc408e460 0x7fddc408e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddcad9d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc408fee0 con 0x7fddc408e460 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddcad9d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40930a0 con 0x7fddc412f9e0 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddcad9d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc408fc60 con 0x7fddc408db50 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 0x7fddc412f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 0x7fddc412f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54158/0 (socket says 172.21.15.177:54158) 2024-09-13T18:34:54.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 -- 172.21.15.177:0/2781626274 learned_addr learned my addr 172.21.15.177:0/2781626274 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc3fff640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc408db50 0x7fddc408df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc933a640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc408e460 0x7fddc408e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 -- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc408db50 msgr2=0x7fddc408df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc408db50 0x7fddc408df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 -- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc408e460 msgr2=0x7fddc408e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc408e460 0x7fddc408e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.227+0000 7fddc8b39640 1 -- 172.21.15.177:0/2781626274 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc408edd0 con 0x7fddc412f9e0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddc8b39640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 0x7fddc412f220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000fd0 tx=0x7fddc4099820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800f9c0 con 0x7fddc412f9e0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800fb40 con 0x7fddc412f9e0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80133f0 con 0x7fddc412f9e0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4090630 con 0x7fddc412f9e0 2024-09-13T18:34:54.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc4140020 con 0x7fddc412f9e0 2024-09-13T18:34:54.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fddb8013a70 con 0x7fddc412f9e0 2024-09-13T18:34:54.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.228+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fddb801d840 con 0x7fddc412f9e0 2024-09-13T18:34:54.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.229+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc405ed70 con 0x7fddc412f9e0 2024-09-13T18:34:54.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.231+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fddb8026490 con 0x7fddc412f9e0 2024-09-13T18:34:54.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.292+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fddc4064490 con 0x7fddc412f9e0 2024-09-13T18:34:54.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.302+0000 7fddc0ff9640 1 -- 172.21.15.177:0/2781626274 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fddb8026630 con 0x7fddc412f9e0 2024-09-13T18:34:54.304 INFO:teuthology.orchestra.run.smithi177.stderr:adjusted tunables profile to default 2024-09-13T18:34:54.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 msgr2=0x7fddc412f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 0x7fddc412f220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000fd0 tx=0x7fddc4099820 comp rx=0 tx=0).stop 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 shutdown_connections 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc408e460 0x7fddc408e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fddc408db50 0x7fddc408df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 --2- 172.21.15.177:0/2781626274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddc412f9e0 0x7fddc412f220 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 >> 172.21.15.177:0/2781626274 conn(0x7fddc4099e10 msgr2=0x7fddc4096810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 shutdown_connections 2024-09-13T18:34:54.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.303+0000 7fddcad9d640 1 -- 172.21.15.177:0/2781626274 wait complete. 2024-09-13T18:34:54.316 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T18:34:54.316 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T18:34:54.316 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:54.316 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T18:34:54.328 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:54.326+0000 7f1a21bbeec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:34:54.341 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:54.341 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T18:34:54.369 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:54.367+0000 7f1a21bbeec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:34:54.393 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:54.393 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T18:34:54.420 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:54.418+0000 7f1a21bbeec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:34:54.444 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:34:54.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T18:34:54.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new f9b65b0a-b05a-4d13-bb43-2376cd9392da 0 2024-09-13T18:34:54.560 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:54.558+0000 7f1a21bbeec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.643+0000 7f2ab1ed4640 1 Processor -- start 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.643+0000 7f2ab1ed4640 1 -- start start 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2ab1ed4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 0x7f2aac0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2ab1ed4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2aac068ae0 0x7f2aac1360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2ab1ed4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac0952d0 con 0x7f2aac0686f0 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2ab1ed4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac069840 con 0x7f2aac068ae0 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2ab1ed4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac094790 con 0x7f2aac096a60 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2aac068ae0 0x7f2aac1360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 0x7f2aac0681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2aac068ae0 0x7f2aac1360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41552/0 (socket says 172.21.15.177:41552) 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 -- 172.21.15.177:0/922579247 learned_addr learned my addr 172.21.15.177:0/922579247 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 0x7f2aac0681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54206/0 (socket says 172.21.15.177:54206) 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 -- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 msgr2=0x7f2aac0681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 --2- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 0x7f2aac0681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 -- 172.21.15.177:0/922579247 >> v1:172.21.15.177:6789/0 conn(0x7f2aac0686f0 legacy=0x7f2aac1329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aabfff640 1 -- 172.21.15.177:0/922579247 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac06a0a0 con 0x7f2aac068ae0 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aaa7fc640 1 -- 172.21.15.177:0/922579247 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924417184 0 0) 0x7f2aac0952d0 con 0x7f2aac0686f0 2024-09-13T18:34:54.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.644+0000 7f2aab7fe640 1 --2- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 0x7f2aac0681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:54.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.645+0000 7f2aabfff640 1 --2- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac068ae0 0x7f2aac1360f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009f20 tx=0x7f2aa4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f646f97e8233a8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2aaa7fc640 1 -- 172.21.15.177:0/922579247 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4031400 con 0x7f2aac068ae0 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2aaa7fc640 1 -- 172.21.15.177:0/922579247 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa402d050 con 0x7f2aac068ae0 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2aaa7fc640 1 -- 172.21.15.177:0/922579247 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4031790 con 0x7f2aac068ae0 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac068ae0 msgr2=0x7f2aac1360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac068ae0 0x7f2aac1360f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009f20 tx=0x7f2aa4009840 comp rx=0 tx=0).stop 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/922579247 shutdown_connections 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac068ae0 0x7f2aac1360f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/922579247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac096a60 0x7f2aac0681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/922579247 >> 172.21.15.177:0/922579247 conn(0x7f2aac097b50 msgr2=0x7f2aac097f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/922579247 shutdown_connections 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.646+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/922579247 wait complete. 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.647+0000 7f2ab1ed4640 1 Processor -- start 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.647+0000 7f2ab1ed4640 1 -- start start 2024-09-13T18:34:54.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2ab1ed4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 0x7f2aac08cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2ab1ed4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aac068ae0 0x7f2aac08d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2ab1ed4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac096a60 0x7f2aac08d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2ab1ed4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac090210 con 0x7f2aac096a60 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2ab1ed4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac08f660 con 0x7f2aac0686f0 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2ab1ed4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac08ee60 con 0x7f2aac068ae0 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 0x7f2aac08cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 0x7f2aac08cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54210/0 (socket says 172.21.15.177:54210) 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aab7fe640 1 -- 172.21.15.177:0/911136132 learned_addr learned my addr 172.21.15.177:0/911136132 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aac068ae0 0x7f2aac08d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aabfff640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac096a60 0x7f2aac08d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 -- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 msgr2=0x7f2aac08cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 0x7f2aac08cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 -- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac096a60 msgr2=0x7f2aac08d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac096a60 0x7f2aac08d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 -- 172.21.15.177:0/911136132 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c009390 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aab7fe640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 0x7f2aac08cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.648+0000 7f2aaaffd640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aac068ae0 0x7f2aac08d010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2aac06acf0 tx=0x7f2aa0007c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0011590 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0011710 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa4009ae0 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0011b10 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac08dd90 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2aa001c440 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.649+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2aa0012770 con 0x7f2aac068ae0 2024-09-13T18:34:54.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.650+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac05ed70 con 0x7f2aac068ae0 2024-09-13T18:34:54.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.653+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f2aa0017070 con 0x7f2aac068ae0 2024-09-13T18:34:54.673 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:54.671+0000 7f1a21bbeec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:34:54.682 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:54.680+0000 7fdffd819ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:34:54.723 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:54.721+0000 7fdffd819ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:34:54.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.721+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd new", "uuid": "f9b65b0a-b05a-4d13-bb43-2376cd9392da", "id": 0} v 0) v1 -- 0x7f2aac096390 con 0x7f2aac068ae0 2024-09-13T18:34:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.732+0000 7f2a937fe640 1 -- 172.21.15.177:0/911136132 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9b65b0a-b05a-4d13-bb43-2376cd9392da", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2aa0007ce0 con 0x7f2aac068ae0 2024-09-13T18:34:54.734 INFO:teuthology.orchestra.run.smithi177.stdout:0 2024-09-13T18:34:54.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aac068ae0 msgr2=0x7f2aac08d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aac068ae0 0x7f2aac08d010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2aac06acf0 tx=0x7f2aa0007c50 comp rx=0 tx=0).stop 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 shutdown_connections 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aac096a60 0x7f2aac08d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aac068ae0 0x7f2aac08d010 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 --2- 172.21.15.177:0/911136132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aac0686f0 0x7f2aac08cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 >> 172.21.15.177:0/911136132 conn(0x7f2aac097b50 msgr2=0x7f2aac06ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 shutdown_connections 2024-09-13T18:34:54.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.733+0000 7f2ab1ed4640 1 -- 172.21.15.177:0/911136132 wait complete. 2024-09-13T18:34:54.749 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new ab422818-9dbe-4e2f-9da7-8a78a041b09b 1 2024-09-13T18:34:54.777 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:54.775+0000 7fdffd819ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:34:54.857 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:54.855+0000 7fdffd819ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:34:54.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.869+0000 7f556fcd4640 1 Processor -- start 2024-09-13T18:34:54.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.869+0000 7f556fcd4640 1 -- start start 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556fcd4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5568095b80 0x7f5568091660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556fcd4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5568091ba0 0x7f5568091f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556fcd4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680924b0 con 0x7f556812b2f0 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556fcd4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5568092bf0 con 0x7f5568091ba0 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556fcd4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5568093890 con 0x7f5568095b80 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556d26f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5568091ba0 0x7f5568091f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556da70640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5568095b80 0x7f5568091660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556e271640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f556812b2f0 0x7f556812d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:50640/0 (socket says 172.21.15.177:50640) 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556e271640 1 -- 172.21.15.177:0/1396318405 learned_addr learned my addr 172.21.15.177:0/1396318405 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556ca6e640 1 -- 172.21.15.177:0/1396318405 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151638771 0 0) 0x7f55680924b0 con 0x7f556812b2f0 2024-09-13T18:34:54.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556ca6e640 1 -- 172.21.15.177:0/1396318405 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55540034e0 con 0x7f556812b2f0 2024-09-13T18:34:54.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556da70640 1 -- 172.21.15.177:0/1396318405 >> v2:172.21.15.177:3300/0 conn(0x7f5568091ba0 msgr2=0x7f5568091f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:54.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556da70640 1 --2- 172.21.15.177:0/1396318405 >> v2:172.21.15.177:3300/0 conn(0x7f5568091ba0 0x7f5568091f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556da70640 1 -- 172.21.15.177:0/1396318405 >> v1:172.21.15.177:6789/0 conn(0x7f556812b2f0 legacy=0x7f556812d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.870+0000 7f556da70640 1 -- 172.21.15.177:0/1396318405 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5568093450 con 0x7f5568095b80 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.871+0000 7f556da70640 1 --2- 172.21.15.177:0/1396318405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5568095b80 0x7f5568091660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5564009850 tx=0x7f55640090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9170cfa444df93c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.871+0000 7f556ca6e640 1 -- 172.21.15.177:0/1396318405 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556402ad00 con 0x7f5568095b80 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.871+0000 7f556ca6e640 1 -- 172.21.15.177:0/1396318405 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556402ae60 con 0x7f5568095b80 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.871+0000 7f556ca6e640 1 -- 172.21.15.177:0/1396318405 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564032670 con 0x7f5568095b80 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 -- 172.21.15.177:0/1396318405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5568095b80 msgr2=0x7f5568091660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1396318405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5568095b80 0x7f5568091660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5564009850 tx=0x7f55640090f0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 -- 172.21.15.177:0/1396318405 shutdown_connections 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1396318405 >> v2:172.21.15.177:3300/0 conn(0x7f5568091ba0 0x7f5568091f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1396318405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5568095b80 0x7f5568091660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 -- 172.21.15.177:0/1396318405 >> 172.21.15.177:0/1396318405 conn(0x7f5568096c70 msgr2=0x7f5568097070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 -- 172.21.15.177:0/1396318405 shutdown_connections 2024-09-13T18:34:54.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.872+0000 7f556fcd4640 1 -- 172.21.15.177:0/1396318405 wait complete. 2024-09-13T18:34:54.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.873+0000 7f556fcd4640 1 Processor -- start 2024-09-13T18:34:54.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 -- start start 2024-09-13T18:34:54.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 0x7f556812ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5568095b80 0x7f5568062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f556812b2f0 0x7f55680632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556808a3a0 con 0x7f5568091ba0 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556812b0c0 con 0x7f556812b2f0 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556fcd4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55680909b0 con 0x7f5568095b80 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556da70640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 0x7f556812ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556e271640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f556812b2f0 0x7f55680632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556da70640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 0x7f556812ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41590/0 (socket says 172.21.15.177:41590) 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556da70640 1 -- 172.21.15.177:0/1066079601 learned_addr learned my addr 172.21.15.177:0/1066079601 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.874+0000 7f556d26f640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5568095b80 0x7f5568062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556da70640 1 -- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5568095b80 msgr2=0x7f5568062d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556da70640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5568095b80 0x7f5568062d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556da70640 1 -- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f556812b2f0 msgr2=0x7f55680632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556da70640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f556812b2f0 0x7f55680632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556da70640 1 -- 172.21.15.177:0/1066079601 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5564009390 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556d26f640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5568095b80 0x7f5568062d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556e271640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f556812b2f0 0x7f55680632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556da70640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 0x7f556812ab80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f556402a630 tx=0x7f5564032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556403b6e0 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556403b840 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556403bc00 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556808baa0 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.875+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556808b460 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.876+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5564039040 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.876+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f5564044e40 con 0x7f5568091ba0 2024-09-13T18:34:54.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.876+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556805ed70 con 0x7f5568091ba0 2024-09-13T18:34:54.882 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.880+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5564049060 con 0x7f5568091ba0 2024-09-13T18:34:54.949 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.947+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ab422818-9dbe-4e2f-9da7-8a78a041b09b", "id": 1} v 0) v1 -- 0x7f556806d420 con 0x7f5568091ba0 2024-09-13T18:34:54.951 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:54.949+0000 7fdffd819ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:34:54.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.957+0000 7f555dffb640 1 -- 172.21.15.177:0/1066079601 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab422818-9dbe-4e2f-9da7-8a78a041b09b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f556403f030 con 0x7f5568091ba0 2024-09-13T18:34:54.959 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.958+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 msgr2=0x7f556812ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.958+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 0x7f556812ab80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f556402a630 tx=0x7f5564032000 comp rx=0 tx=0).stop 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.958+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 shutdown_connections 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.958+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f556812b2f0 0x7f55680632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.958+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5568095b80 0x7f5568062d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.958+0000 7f556fcd4640 1 --2- 172.21.15.177:0/1066079601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5568091ba0 0x7f556812ab80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.959+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 >> 172.21.15.177:0/1066079601 conn(0x7f5568096c70 msgr2=0x7f556806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.959+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 shutdown_connections 2024-09-13T18:34:54.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:54.959+0000 7f556fcd4640 1 -- 172.21.15.177:0/1066079601 wait complete. 2024-09-13T18:34:54.965 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:54.963+0000 7f1a21bbeec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:34:54.971 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new d0ca7945-14cb-40ee-b8a7-14586c366c5d 2 2024-09-13T18:34:55.020 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.018+0000 7f1a21bbeec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:34:55.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.063+0000 7f293e680640 1 Processor -- start 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.063+0000 7f293e680640 1 -- start start 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293e680640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938095190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293e680640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938132070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293e680640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2938093710 con 0x7f29380956d0 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293e680640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293806a4e0 con 0x7f2938095aa0 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293e680640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f293806e430 con 0x7f2938093ad0 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f2937fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938095190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f2937fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938095190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:47254/0 (socket says 172.21.15.177:47254) 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f2937fff640 1 -- 172.21.15.177:0/4279688041 learned_addr learned my addr 172.21.15.177:0/4279688041 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293cc1d640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938132070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293cc1d640 1 -- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 msgr2=0x7f2938095190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293cc1d640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938095190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293cc1d640 1 -- 172.21.15.177:0/4279688041 >> v1:172.21.15.177:6789/0 conn(0x7f29380956d0 legacy=0x7f293812e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f293cc1d640 1 -- 172.21.15.177:0/4279688041 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938096fc0 con 0x7f2938095aa0 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.064+0000 7f2937fff640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938095190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.065+0000 7f293cc1d640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938132070 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2930009e90 tx=0x7f2930009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc25bdc6f30922d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.065+0000 7f2936ffd640 1 -- 172.21.15.177:0/4279688041 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930031400 con 0x7f2938095aa0 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.065+0000 7f2936ffd640 1 -- 172.21.15.177:0/4279688041 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293002d050 con 0x7f2938095aa0 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.065+0000 7f2936ffd640 1 -- 172.21.15.177:0/4279688041 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930031790 con 0x7f2938095aa0 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 -- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 msgr2=0x7f2938132070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938132070 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2930009e90 tx=0x7f2930009780 comp rx=0 tx=0).stop 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 -- 172.21.15.177:0/4279688041 shutdown_connections 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938132070 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 --2- 172.21.15.177:0/4279688041 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938095190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 -- 172.21.15.177:0/4279688041 >> 172.21.15.177:0/4279688041 conn(0x7f293809a1a0 msgr2=0x7f2938096280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 -- 172.21.15.177:0/4279688041 shutdown_connections 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.066+0000 7f293e680640 1 -- 172.21.15.177:0/4279688041 wait complete. 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.067+0000 7f293e680640 1 Processor -- start 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.067+0000 7f293e680640 1 -- start start 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293e680640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938090600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293e680640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29380956d0 0x7f2938090b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293e680640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938091080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293e680640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938093210 con 0x7f29380956d0 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293e680640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380916b0 con 0x7f2938095aa0 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293e680640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f293811ecd0 con 0x7f2938093ad0 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f2937fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938090600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f2937fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938090600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:47270/0 (socket says 172.21.15.177:47270) 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f2937fff640 1 -- 172.21.15.177:0/1771240903 learned_addr learned my addr 172.21.15.177:0/1771240903 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:55.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f29377fe640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29380956d0 0x7f2938090b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f293cc1d640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938091080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.068+0000 7f2937fff640 1 -- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 msgr2=0x7f2938091080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f2937fff640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938091080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f2937fff640 1 -- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29380956d0 msgr2=0x7f2938090b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f2937fff640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29380956d0 0x7f2938090b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f2937fff640 1 -- 172.21.15.177:0/1771240903 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928009450 con 0x7f2938093ad0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f29377fe640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29380956d0 0x7f2938090b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f2937fff640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938090600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f292800ec80 tx=0x7f29280091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928009730 con 0x7f2938093ad0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292800eec0 con 0x7f2938093ad0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292800cdf0 con 0x7f2938093ad0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2930009a20 con 0x7f2938093ad0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2938092570 con 0x7f2938093ad0 2024-09-13T18:34:55.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2928005970 con 0x7f2938093ad0 2024-09-13T18:34:55.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.069+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f2928013e20 con 0x7f2938093ad0 2024-09-13T18:34:55.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.070+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293805ed70 con 0x7f2938093ad0 2024-09-13T18:34:55.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.074+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f2928021060 con 0x7f2938093ad0 2024-09-13T18:34:55.182 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.180+0000 7fdffd819ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:34:55.194 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.193+0000 7f1a21bbeec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:34:55.204 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.202+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd new", "uuid": "d0ca7945-14cb-40ee-b8a7-14586c366c5d", "id": 2} v 0) v1 -- 0x7f29380681b0 con 0x7f2938093ad0 2024-09-13T18:34:55.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.212+0000 7f291ffff640 1 -- 172.21.15.177:0/1771240903 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d0ca7945-14cb-40ee-b8a7-14586c366c5d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2928017070 con 0x7f2938093ad0 2024-09-13T18:34:55.214 INFO:teuthology.orchestra.run.smithi177.stdout:2 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 msgr2=0x7f2938090600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938090600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f292800ec80 tx=0x7f29280091d0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 shutdown_connections 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938095aa0 0x7f2938091080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29380956d0 0x7f2938090b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 --2- 172.21.15.177:0/1771240903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2938093ad0 0x7f2938090600 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.213+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 >> 172.21.15.177:0/1771240903 conn(0x7f293809a1a0 msgr2=0x7f293806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.214+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 shutdown_connections 2024-09-13T18:34:55.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.214+0000 7f293e680640 1 -- 172.21.15.177:0/1771240903 wait complete. 2024-09-13T18:34:55.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 63594787-3078-431f-9ac4-e24e4bac0e7d 3 2024-09-13T18:34:55.229 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.227+0000 7fdffd819ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:34:55.320 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.318+0000 7fdffd819ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:34:55.321 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 Processor -- start 2024-09-13T18:34:55.321 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 -- start start 2024-09-13T18:34:55.322 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 0x7f5e5012a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.322 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012ac70 0x7f5e50097b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.322 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e50098040 0x7f5e50098490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.322 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e50069820 con 0x7f5e50098040 2024-09-13T18:34:55.322 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50094280 con 0x7f5e5012ac70 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.319+0000 7f5e56f9d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50068c80 con 0x7f5e5012a360 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 0x7f5e5012a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 0x7f5e5012a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:47272/0 (socket says 172.21.15.177:47272) 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 -- 172.21.15.177:0/2190587251 learned_addr learned my addr 172.21.15.177:0/2190587251 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e5553a640 1 --2- 172.21.15.177:0/2190587251 >> v2:172.21.15.177:3300/0 conn(0x7f5e50098040 0x7f5e50098490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e4ffff640 1 --2- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012ac70 0x7f5e50097b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 -- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012ac70 msgr2=0x7f5e50097b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 --2- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012ac70 0x7f5e50097b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 -- 172.21.15.177:0/2190587251 >> v2:172.21.15.177:3300/0 conn(0x7f5e50098040 msgr2=0x7f5e50098490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 --2- 172.21.15.177:0/2190587251 >> v2:172.21.15.177:3300/0 conn(0x7f5e50098040 0x7f5e50098490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e54d39640 1 -- 172.21.15.177:0/2190587251 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50069050 con 0x7f5e5012a360 2024-09-13T18:34:55.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.320+0000 7f5e5553a640 1 --2- 172.21.15.177:0/2190587251 >> v2:172.21.15.177:3300/0 conn(0x7f5e50098040 0x7f5e50098490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.321+0000 7f5e54d39640 1 --2- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 0x7f5e5012a730 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009940 tx=0x7f5e440091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eb2c5ed7af7790 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.321+0000 7f5e4f7fe640 1 -- 172.21.15.177:0/2190587251 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4402ac70 con 0x7f5e5012a360 2024-09-13T18:34:55.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.321+0000 7f5e4f7fe640 1 -- 172.21.15.177:0/2190587251 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4402add0 con 0x7f5e5012a360 2024-09-13T18:34:55.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.321+0000 7f5e4f7fe640 1 -- 172.21.15.177:0/2190587251 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44032670 con 0x7f5e5012a360 2024-09-13T18:34:55.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.321+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 msgr2=0x7f5e5012a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.321+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 0x7f5e5012a730 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009940 tx=0x7f5e440091d0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2190587251 shutdown_connections 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2190587251 >> v2:172.21.15.177:3300/0 conn(0x7f5e50098040 0x7f5e50098490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012ac70 0x7f5e50097b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2190587251 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e5012a360 0x7f5e5012a730 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2190587251 >> 172.21.15.177:0/2190587251 conn(0x7f5e50090be0 msgr2=0x7f5e5009d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2190587251 shutdown_connections 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2190587251 wait complete. 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.322+0000 7f5e56f9d640 1 Processor -- start 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 -- start start 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e50098040 0x7f5e501395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012a360 0x7f5e50139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 0x7f5e5008be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5008e180 con 0x7f5e5012ac70 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5008f580 con 0x7f5e5012a360 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e56f9d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5008d880 con 0x7f5e50098040 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e4ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012a360 0x7f5e50139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 0x7f5e5008be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e54d39640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e50098040 0x7f5e501395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e4ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012a360 0x7f5e50139b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54260/0 (socket says 172.21.15.177:54260) 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 0x7f5e5008be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41612/0 (socket says 172.21.15.177:41612) 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e4ffff640 1 -- 172.21.15.177:0/2220668971 learned_addr learned my addr 172.21.15.177:0/2220668971 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 -- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e50098040 msgr2=0x7f5e501395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e50098040 0x7f5e501395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 -- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012a360 msgr2=0x7f5e50139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012a360 0x7f5e50139b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e5553a640 1 -- 172.21.15.177:0/2220668971 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e48009a20 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.323+0000 7f5e54d39640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e50098040 0x7f5e501395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e5553a640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 0x7f5e5008be70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009e70 tx=0x7f5e4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800cd40 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4800f040 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e44009450 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e480146d0 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5013a1e0 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5e48003020 con 0x7f5e5012ac70 2024-09-13T18:34:55.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.324+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f5e4801d410 con 0x7f5e5012ac70 2024-09-13T18:34:55.330 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.325+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5005ed70 con 0x7f5e5012ac70 2024-09-13T18:34:55.330 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.328+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5e48018030 con 0x7f5e5012ac70 2024-09-13T18:34:55.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.396+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "63594787-3078-431f-9ac4-e24e4bac0e7d", "id": 3} v 0) v1 -- 0x7f5e500643f0 con 0x7f5e5012ac70 2024-09-13T18:34:55.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.401+0000 7f5e4cff9640 1 -- 172.21.15.177:0/2220668971 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63594787-3078-431f-9ac4-e24e4bac0e7d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5e48014db0 con 0x7f5e5012ac70 2024-09-13T18:34:55.403 INFO:teuthology.orchestra.run.smithi177.stdout:3 2024-09-13T18:34:55.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 msgr2=0x7f5e5008be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 0x7f5e5008be70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009e70 tx=0x7f5e4800b040 comp rx=0 tx=0).stop 2024-09-13T18:34:55.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 shutdown_connections 2024-09-13T18:34:55.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e5012ac70 0x7f5e5008be70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e5012a360 0x7f5e50139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 --2- 172.21.15.177:0/2220668971 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e50098040 0x7f5e501395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 >> 172.21.15.177:0/2220668971 conn(0x7f5e50090be0 msgr2=0x7f5e50091250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 shutdown_connections 2024-09-13T18:34:55.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.402+0000 7f5e56f9d640 1 -- 172.21.15.177:0/2220668971 wait complete. 2024-09-13T18:34:55.414 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T18:34:55.415 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T18:34:55.417 INFO:tasks.ceph.osd.0:Started 2024-09-13T18:34:55.417 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T18:34:55.417 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T18:34:55.419 INFO:tasks.ceph.osd.1:Started 2024-09-13T18:34:55.419 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T18:34:55.419 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T18:34:55.421 INFO:tasks.ceph.osd.2:Started 2024-09-13T18:34:55.422 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T18:34:55.422 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T18:34:55.424 INFO:tasks.ceph.osd.3:Started 2024-09-13T18:34:55.424 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:34:55.567 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.564+0000 7f5581edd640 1 Processor -- start 2024-09-13T18:34:55.567 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.564+0000 7f5581edd640 1 -- start start 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f5581edd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 0x7f557c091440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f5581edd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c12e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f5581edd640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c0931a0 con 0x7f557c091980 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 0x7f557c091440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f5581edd640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0966e0 con 0x7f557c095960 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f5581edd640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0933f0 con 0x7f557c091d50 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 0x7f557c091440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54284/0 (socket says 172.21.15.177:54284) 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557b7fe640 1 -- 172.21.15.177:0/3382915708 learned_addr learned my addr 172.21.15.177:0/3382915708 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557bfff640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c12e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557bfff640 1 -- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 msgr2=0x7f557c091440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557bfff640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 0x7f557c091440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557bfff640 1 -- 172.21.15.177:0/3382915708 >> v1:172.21.15.177:6789/0 conn(0x7f557c091980 legacy=0x7f557c12b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.569 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557bfff640 1 -- 172.21.15.177:0/3382915708 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0921d0 con 0x7f557c091d50 2024-09-13T18:34:55.569 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.565+0000 7f557b7fe640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 0x7f557c091440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.566+0000 7f557bfff640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c12e800 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f557400bf30 tx=0x7f5574009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2cca5fe483696b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.566+0000 7f557a7fc640 1 -- 172.21.15.177:0/3382915708 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557400b2e0 con 0x7f557c091d50 2024-09-13T18:34:55.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.566+0000 7f557a7fc640 1 -- 172.21.15.177:0/3382915708 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5574009550 con 0x7f557c091d50 2024-09-13T18:34:55.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.566+0000 7f557a7fc640 1 -- 172.21.15.177:0/3382915708 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574009930 con 0x7f557c091d50 2024-09-13T18:34:55.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 -- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 msgr2=0x7f557c12e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c12e800 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f557400bf30 tx=0x7f5574009310 comp rx=0 tx=0).stop 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 -- 172.21.15.177:0/3382915708 shutdown_connections 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c12e800 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 --2- 172.21.15.177:0/3382915708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c095960 0x7f557c091440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 -- 172.21.15.177:0/3382915708 >> 172.21.15.177:0/3382915708 conn(0x7f557c09a9b0 msgr2=0x7f557c096a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 -- 172.21.15.177:0/3382915708 shutdown_connections 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.567+0000 7f5581edd640 1 -- 172.21.15.177:0/3382915708 wait complete. 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.568+0000 7f5581edd640 1 Processor -- start 2024-09-13T18:34:55.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.568+0000 7f5581edd640 1 -- start start 2024-09-13T18:34:55.573 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 0x7f557c08c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.573 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c08cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.573 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f557c095960 0x7f557c08d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.573 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c090240 con 0x7f557c095960 2024-09-13T18:34:55.573 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c08e280 con 0x7f557c091980 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f557c08ff80 con 0x7f557c091d50 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 0x7f557c08c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 0x7f557c08c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54300/0 (socket says 172.21.15.177:54300) 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 -- 172.21.15.177:0/39349592 learned_addr learned my addr 172.21.15.177:0/39349592 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557affd640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c08cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557bfff640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f557c095960 0x7f557c08d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 -- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 msgr2=0x7f557c08cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c08cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 -- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f557c095960 msgr2=0x7f557c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f557c095960 0x7f557c08d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 -- 172.21.15.177:0/39349592 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5570009450 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557affd640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c08cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557bfff640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f557c095960 0x7f557c08d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f557b7fe640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 0x7f557c08c750 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f557000ec80 tx=0x7f55700091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570009730 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557000eec0 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557400ba40 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.569+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000cdd0 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.570+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c090bb0 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.570+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5570005970 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.570+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f5570005c20 con 0x7f557c091980 2024-09-13T18:34:55.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.571+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557c08fc40 con 0x7f557c091980 2024-09-13T18:34:55.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.573+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5570021060 con 0x7f557c091980 2024-09-13T18:34:55.596 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.594+0000 7f1a21bbeec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:34:55.620 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.618+0000 7fdffd819ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:34:55.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.652+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f557c090e30 con 0x7f557c091980 2024-09-13T18:34:55.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.653+0000 7f55637fe640 1 -- 172.21.15.177:0/39349592 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f5570029020 con 0x7f557c091980 2024-09-13T18:34:55.655 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:34:55.655 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":6,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:34:55.397217+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 msgr2=0x7f557c08c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 0x7f557c08c750 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f557000ec80 tx=0x7f55700091d0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 shutdown_connections 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f557c095960 0x7f557c08d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f557c091d50 0x7f557c08cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 --2- 172.21.15.177:0/39349592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f557c091980 0x7f557c08c750 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 >> 172.21.15.177:0/39349592 conn(0x7f557c09a9b0 msgr2=0x7f557c06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 shutdown_connections 2024-09-13T18:34:55.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:34:55.654+0000 7f5581edd640 1 -- 172.21.15.177:0/39349592 wait complete. 2024-09-13T18:34:55.657 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.655+0000 7f1a21bbeec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:34:55.664 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.662+0000 7fdffd819ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:34:55.666 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T18:34:55.667 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T18:34:55.742 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.740+0000 7f1a21bbeec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:34:55.745 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.744+0000 7fdffd819ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:34:55.819 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.817+0000 7f1a21bbeec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:34:55.822 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.820+0000 7fdffd819ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:34:55.864 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.862+0000 7f1a21bbeec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:34:55.867 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.865+0000 7fdffd819ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:34:55.945 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.943+0000 7f1a21bbeec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:34:55.945 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.944+0000 7fdffd819ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:34:55.997 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:55.996+0000 7fdffd819ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:34:55.998 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:55.996+0000 7f1a21bbeec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:34:56.037 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:34:56.035+0000 7f9db7b23440 -1 Falling back to public interface 2024-09-13T18:34:56.042 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:34:56.039+0000 7f8d21e42440 -1 Falling back to public interface 2024-09-13T18:34:56.042 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:34:56.039+0000 7fd69e65a440 -1 Falling back to public interface 2024-09-13T18:34:56.042 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:34:56.039+0000 7f8d1fdfd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6804/0: (98) Address already in use 2024-09-13T18:34:56.043 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:34:56.040+0000 7fd69cffd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6808/0: (98) Address already in use 2024-09-13T18:34:56.043 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:34:56.040+0000 7f8d1fdfd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6810/0: (98) Address already in use 2024-09-13T18:34:56.043 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.040+0000 7fdffd819ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:34:56.044 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.043+0000 7f1a21bbeec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:34:56.046 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:34:56.044+0000 7f5dd217d440 -1 Falling back to public interface 2024-09-13T18:34:56.070 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:34:56.139 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.137+0000 7f1a21bbeec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:34:56.139 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.137+0000 7fdffd819ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:34:56.156 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.154+0000 7fbd430dd640 1 Processor -- start 2024-09-13T18:34:56.156 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.154+0000 7fbd430dd640 1 -- start start 2024-09-13T18:34:56.157 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd430dd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 0x7fbd3c12cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd430dd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbd3c12d2d0 0x7fbd3c12f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd430dd640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c09a360 con 0x7fbd3c097a80 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd430dd640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c09b3f0 con 0x7fbd3c12d2d0 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd430dd640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c09b6b0 con 0x7fbd3c098560 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 0x7fbd3c12cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd4167a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbd3c12d2d0 0x7fbd3c12f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 0x7fbd3c12cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54304/0 (socket says 172.21.15.177:54304) 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 -- 172.21.15.177:0/1791493336 learned_addr learned my addr 172.21.15.177:0/1791493336 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 -- 172.21.15.177:0/1791493336 >> v2:172.21.15.177:3300/0 conn(0x7fbd3c12d2d0 msgr2=0x7fbd3c12f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:56.158 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 --2- 172.21.15.177:0/1791493336 >> v2:172.21.15.177:3300/0 conn(0x7fbd3c12d2d0 0x7fbd3c12f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.159 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 -- 172.21.15.177:0/1791493336 >> v1:172.21.15.177:6789/0 conn(0x7fbd3c097a80 legacy=0x7fbd3c097e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:56.159 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3b7fe640 1 -- 172.21.15.177:0/1791493336 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951496263 0 0) 0x7fbd3c09a360 con 0x7fbd3c097a80 2024-09-13T18:34:56.159 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd3bfff640 1 -- 172.21.15.177:0/1791493336 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c09ab20 con 0x7fbd3c098560 2024-09-13T18:34:56.159 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.155+0000 7fbd4167a640 1 --2- 172.21.15.177:0/1791493336 >> v2:172.21.15.177:3300/0 conn(0x7fbd3c12d2d0 0x7fbd3c12f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.156+0000 7fbd3bfff640 1 --2- 172.21.15.177:0/1791493336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 0x7fbd3c12cd90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009910 tx=0x7fbd280091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a0ef2ceeecfa26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.156+0000 7fbd3b7fe640 1 -- 172.21.15.177:0/1791493336 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2802add0 con 0x7fbd3c098560 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.156+0000 7fbd3b7fe640 1 -- 172.21.15.177:0/1791493336 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28032440 con 0x7fbd3c098560 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.156+0000 7fbd3b7fe640 1 -- 172.21.15.177:0/1791493336 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd280327e0 con 0x7fbd3c098560 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 -- 172.21.15.177:0/1791493336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 msgr2=0x7fbd3c12cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 --2- 172.21.15.177:0/1791493336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 0x7fbd3c12cd90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009910 tx=0x7fbd280091d0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 -- 172.21.15.177:0/1791493336 shutdown_connections 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 --2- 172.21.15.177:0/1791493336 >> v2:172.21.15.177:3300/0 conn(0x7fbd3c12d2d0 0x7fbd3c12f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 --2- 172.21.15.177:0/1791493336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c098560 0x7fbd3c12cd90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 -- 172.21.15.177:0/1791493336 >> 172.21.15.177:0/1791493336 conn(0x7fbd3c06baa0 msgr2=0x7fbd3c06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 -- 172.21.15.177:0/1791493336 shutdown_connections 2024-09-13T18:34:56.161 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.157+0000 7fbd430dd640 1 -- 172.21.15.177:0/1791493336 wait complete. 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 Processor -- start 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 -- start start 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbd3c097a80 0x7fbd3c069af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 0x7fbd3c06a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 0x7fbd3c139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1247f0 con 0x7fbd3c098560 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c06ab00 con 0x7fbd3c12d2d0 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd430dd640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c116e60 con 0x7fbd3c097a80 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 0x7fbd3c06a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd4167a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 0x7fbd3c139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 0x7fbd3c06a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41676/0 (socket says 172.21.15.177:41676) 2024-09-13T18:34:56.162 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd3bfff640 1 -- 172.21.15.177:0/2296299253 learned_addr learned my addr 172.21.15.177:0/2296299253 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd4167a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 0x7fbd3c139120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54306/0 (socket says 172.21.15.177:54306) 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.158+0000 7fbd40e79640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbd3c097a80 0x7fbd3c069af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd3bfff640 1 -- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbd3c097a80 msgr2=0x7fbd3c069af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd3bfff640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbd3c097a80 0x7fbd3c069af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd3bfff640 1 -- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 msgr2=0x7fbd3c139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd3bfff640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 0x7fbd3c139120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd3bfff640 1 -- 172.21.15.177:0/2296299253 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009a20 con 0x7fbd3c098560 2024-09-13T18:34:56.163 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd4167a640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 0x7fbd3c139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:56.164 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd3bfff640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 0x7fbd3c06a030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbd2802a700 tx=0x7fbd28032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:56.164 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28031bb0 con 0x7fbd3c098560 2024-09-13T18:34:56.164 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28031d30 con 0x7fbd3c098560 2024-09-13T18:34:56.164 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28009450 con 0x7fbd3c098560 2024-09-13T18:34:56.165 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28031ea0 con 0x7fbd3c098560 2024-09-13T18:34:56.165 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c116870 con 0x7fbd3c098560 2024-09-13T18:34:56.165 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fbd28039040 con 0x7fbd3c098560 2024-09-13T18:34:56.165 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.159+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fbd2803b900 con 0x7fbd3c098560 2024-09-13T18:34:56.165 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.160+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c05ed70 con 0x7fbd3c098560 2024-09-13T18:34:56.165 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.162+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fbd28042070 con 0x7fbd3c098560 2024-09-13T18:34:56.191 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.189+0000 7f1a21bbeec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:34:56.191 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.189+0000 7fdffd819ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:34:56.221 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.219+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd3c067130 con 0x7fbd3c098560 2024-09-13T18:34:56.224 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.222+0000 7fbd38ff9640 1 -- 172.21.15.177:0/2296299253 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fbd28037070 con 0x7fbd3c098560 2024-09-13T18:34:56.224 INFO:teuthology.misc.health.smithi177.stdout: 2024-09-13T18:34:56.224 INFO:teuthology.misc.health.smithi177.stdout:{"epoch":6,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:34:55.397217+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:34:56.225 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 msgr2=0x7fbd3c06a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:56.225 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 0x7fbd3c06a030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbd2802a700 tx=0x7fbd28032000 comp rx=0 tx=0).stop 2024-09-13T18:34:56.225 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 shutdown_connections 2024-09-13T18:34:56.225 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3c12d2d0 0x7fbd3c139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.226 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3c098560 0x7fbd3c06a030 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.226 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 --2- 172.21.15.177:0/2296299253 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbd3c097a80 0x7fbd3c069af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:56.226 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 >> 172.21.15.177:0/2296299253 conn(0x7fbd3c06baa0 msgr2=0x7fbd3c12e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:56.226 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 shutdown_connections 2024-09-13T18:34:56.226 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:34:56.223+0000 7fbd430dd640 1 -- 172.21.15.177:0/2296299253 wait complete. 2024-09-13T18:34:56.235 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T18:34:56.245 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.243+0000 7fdffd819ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:34:56.245 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.243+0000 7f1a21bbeec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:34:56.314 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.311+0000 7f1a21bbeec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:34:56.314 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.312+0000 7fdffd819ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:34:56.358 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.356+0000 7fdffd819ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:34:56.359 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.356+0000 7f1a21bbeec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:34:56.401 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.399+0000 7fdffd819ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:34:56.401 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.399+0000 7f1a21bbeec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:34:56.604 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:56.602+0000 7fdffd819ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:34:56.605 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:34:56.602+0000 7f1a21bbeec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:34:57.619 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:57.616+0000 7fdfe6cb3640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T18:34:57.619 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:34:57.617+0000 7fdfe6cb3640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T18:34:59.482 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:34:59.480+0000 7f8d21e42440 -1 osd.2 0 log_to_monitors true 2024-09-13T18:34:59.560 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:34:59.558+0000 7f5dd217d440 -1 osd.1 0 log_to_monitors true 2024-09-13T18:34:59.563 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:34:59.561+0000 7f9db7b23440 -1 osd.3 0 log_to_monitors true 2024-09-13T18:34:59.573 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:34:59.571+0000 7fd69e65a440 -1 osd.0 0 log_to_monitors true 2024-09-13T18:35:02.616 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:35:02.614+0000 7fd6997f6640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T18:35:02.626 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:35:02.624+0000 7fd692684640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:35:02.638 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:35:02.646 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:35:02.645+0000 7f8d1c5f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T18:35:02.652 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:35:02.650+0000 7f5dccbf6640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T18:35:02.653 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:35:02.651+0000 7f8d155e8640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:35:02.659 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:35:02.657+0000 7f5dc5be8640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:35:02.708 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:35:02.706+0000 7f9db25f6640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T18:35:02.713 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:35:02.711+0000 7f9dab5e8640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:35:02.730 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.728+0000 7f786be2b640 1 Processor -- start 2024-09-13T18:35:02.730 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.728+0000 7f786be2b640 1 -- start start 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.728+0000 7f786be2b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786406a8a0 0x7f78641268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786be2b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864127210 0x7f786412a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786be2b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7864112820 con 0x7f7864126e00 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786be2b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7864111b20 con 0x7f786406a8a0 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786be2b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7864111e70 con 0x7f7864127210 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f78693c6640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7864126e00 0x7f7864129b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47424/0 (socket says 172.21.15.177:47424) 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786a3c8640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864127210 0x7f786412a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f7869bc7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786406a8a0 0x7f78641268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f78693c6640 1 -- 172.21.15.177:0/4184128102 learned_addr learned my addr 172.21.15.177:0/4184128102 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786a3c8640 1 -- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786406a8a0 msgr2=0x7f78641268c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:02.732 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f7868bc5640 1 -- 172.21.15.177:0/4184128102 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423861306 0 0) 0x7f7864112820 con 0x7f7864126e00 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786a3c8640 1 --2- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786406a8a0 0x7f78641268c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786a3c8640 1 -- 172.21.15.177:0/4184128102 >> v1:172.21.15.177:6789/0 conn(0x7f7864126e00 legacy=0x7f7864129b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f786a3c8640 1 -- 172.21.15.177:0/4184128102 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78641121a0 con 0x7f7864127210 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.729+0000 7f7869bc7640 1 --2- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786406a8a0 0x7f78641268c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786a3c8640 1 --2- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864127210 0x7f786412a290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f785c009e90 tx=0x7f785c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca82eaff8844d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f7868bc5640 1 -- 172.21.15.177:0/4184128102 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c007550 con 0x7f7864127210 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f7868bc5640 1 -- 172.21.15.177:0/4184128102 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c02d050 con 0x7f7864127210 2024-09-13T18:35:02.733 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f7868bc5640 1 -- 172.21.15.177:0/4184128102 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0078e0 con 0x7f7864127210 2024-09-13T18:35:02.737 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 -- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864127210 msgr2=0x7f786412a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.737 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 --2- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864127210 0x7f786412a290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f785c009e90 tx=0x7f785c009780 comp rx=0 tx=0).stop 2024-09-13T18:35:02.737 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 -- 172.21.15.177:0/4184128102 shutdown_connections 2024-09-13T18:35:02.737 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 --2- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864127210 0x7f786412a290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.737 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 --2- 172.21.15.177:0/4184128102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786406a8a0 0x7f78641268c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.737 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 -- 172.21.15.177:0/4184128102 >> 172.21.15.177:0/4184128102 conn(0x7f786406baa0 msgr2=0x7f786406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 -- 172.21.15.177:0/4184128102 shutdown_connections 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.730+0000 7f786be2b640 1 -- 172.21.15.177:0/4184128102 wait complete. 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.731+0000 7f786be2b640 1 Processor -- start 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.731+0000 7f786be2b640 1 -- start start 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 0x7f786412cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864126e00 0x7f786412d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864127210 0x7f786412da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864130520 con 0x7f786406a8a0 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786412e820 con 0x7f7864127210 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7864130150 con 0x7f7864126e00 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 0x7f786412cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 0x7f786412cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54414/0 (socket says 172.21.15.177:54414) 2024-09-13T18:35:02.738 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f78693c6640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864126e00 0x7f786412d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 -- 172.21.15.177:0/1132406131 learned_addr learned my addr 172.21.15.177:0/1132406131 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786a3c8640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864127210 0x7f786412da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 -- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864126e00 msgr2=0x7f786412d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864126e00 0x7f786412d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 -- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864127210 msgr2=0x7f786412da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864127210 0x7f786412da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 -- 172.21.15.177:0/1132406131 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860009390 con 0x7f786406a8a0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f78693c6640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864126e00 0x7f786412d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7869bc7640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 0x7f786412cff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f786000ec80 tx=0x7f78600090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000eec0 con 0x7f786406a8a0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786000cbc0 con 0x7f786406a8a0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c009a20 con 0x7f786406a8a0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.732+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860013470 con 0x7f786406a8a0 2024-09-13T18:35:02.739 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.733+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78641307e0 con 0x7f786406a8a0 2024-09-13T18:35:02.740 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.734+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786405ed70 con 0x7f786406a8a0 2024-09-13T18:35:02.740 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.734+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 4) v1 ==== 58495+0+0 (secure 0 0 0) 0x7f78600137e0 con 0x7f786406a8a0 2024-09-13T18:35:02.740 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.734+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1509+0+0 (secure 0 0 0) 0x7f7860063020 con 0x7f786406a8a0 2024-09-13T18:35:02.740 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.737+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f786003e710 con 0x7f786406a8a0 2024-09-13T18:35:02.870 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.868+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7864120eb0 con 0x7f786406a8a0 2024-09-13T18:35:02.871 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.869+0000 7f7859ffb640 1 -- 172.21.15.177:0/1132406131 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f7860041f20 con 0x7f786406a8a0 2024-09-13T18:35:02.871 INFO:teuthology.misc.health.smithi177.stdout: 2024-09-13T18:35:02.871 INFO:teuthology.misc.health.smithi177.stdout:{"epoch":8,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:35:00.627574+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 >> v1:172.21.15.177:6816/32265 conn(0x7f784804cae0 legacy=0x7f784804ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 msgr2=0x7f786412cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 0x7f786412cff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f786000ec80 tx=0x7f78600090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 shutdown_connections 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864127210 0x7f786412da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7864126e00 0x7f786412d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 --2- 172.21.15.177:0/1132406131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786406a8a0 0x7f786412cff0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.870+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 >> 172.21.15.177:0/1132406131 conn(0x7f786406baa0 msgr2=0x7f7864129970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.871+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 shutdown_connections 2024-09-13T18:35:02.873 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:02.871+0000 7f786be2b640 1 -- 172.21.15.177:0/1132406131 wait complete. 2024-09-13T18:35:02.881 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T18:35:09.285 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:35:09.381 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.378+0000 7fbf85b9d640 1 Processor -- start 2024-09-13T18:35:09.381 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.378+0000 7fbf85b9d640 1 -- start start 2024-09-13T18:35:09.381 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf85b9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbf80098560 0x7fbf8012ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf85b9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf8012f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf85b9d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf80099bf0 con 0x7fbf80097a80 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf85b9d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf80098950 con 0x7fbf80098560 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf85b9d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8009b3e0 con 0x7fbf8012d230 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf8012f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7effd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbf80098560 0x7fbf8012ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7f7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbf80097a80 0x7fbf80097e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47118/0 (socket says 172.21.15.177:47118) 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf8012f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41408/0 (socket says 172.21.15.177:41408) 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7effd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbf80098560 0x7fbf8012ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41390/0 (socket says 172.21.15.177:41390) 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7ffff640 1 -- 172.21.15.177:0/447817501 learned_addr learned my addr 172.21.15.177:0/447817501 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7effd640 1 -- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 msgr2=0x7fbf8012f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:09.382 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7effd640 1 --2- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf8012f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.383 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7effd640 1 -- 172.21.15.177:0/447817501 >> v1:172.21.15.177:6789/0 conn(0x7fbf80097a80 legacy=0x7fbf80097e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.383 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7effd640 1 -- 172.21.15.177:0/447817501 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8009b0d0 con 0x7fbf80098560 2024-09-13T18:35:09.383 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7e7fc640 1 -- 172.21.15.177:0/447817501 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993554560 0 0) 0x7fbf80099bf0 con 0x7fbf80097a80 2024-09-13T18:35:09.383 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.379+0000 7fbf7ffff640 1 --2- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf8012f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:09.386 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.380+0000 7fbf7effd640 1 --2- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80098560 0x7fbf8012ccf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009850 tx=0x7fbf740090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109253252ad84a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:09.386 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.380+0000 7fbf7e7fc640 1 -- 172.21.15.177:0/447817501 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7402ac80 con 0x7fbf80098560 2024-09-13T18:35:09.386 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.380+0000 7fbf7e7fc640 1 -- 172.21.15.177:0/447817501 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7402ade0 con 0x7fbf80098560 2024-09-13T18:35:09.386 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.380+0000 7fbf7e7fc640 1 -- 172.21.15.177:0/447817501 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74032670 con 0x7fbf80098560 2024-09-13T18:35:09.386 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 -- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80098560 msgr2=0x7fbf8012ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80098560 0x7fbf8012ccf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009850 tx=0x7fbf740090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 -- 172.21.15.177:0/447817501 shutdown_connections 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf8012f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/447817501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80098560 0x7fbf8012ccf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 -- 172.21.15.177:0/447817501 >> 172.21.15.177:0/447817501 conn(0x7fbf8006baa0 msgr2=0x7fbf8006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 -- 172.21.15.177:0/447817501 shutdown_connections 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.381+0000 7fbf85b9d640 1 -- 172.21.15.177:0/447817501 wait complete. 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 Processor -- start 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 -- start start 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 0x7fbf80136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.387 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbf80098560 0x7fbf80137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf80137990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80118fb0 con 0x7fbf80097a80 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf801172b0 con 0x7fbf8012d230 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf85b9d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80118be0 con 0x7fbf80098560 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 0x7fbf80136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 0x7fbf80136f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41404/0 (socket says 172.21.15.177:41404) 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf7effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbf80098560 0x7fbf80137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf7f7fe640 1 -- 172.21.15.177:0/2173579272 learned_addr learned my addr 172.21.15.177:0/2173579272 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.382+0000 7fbf7ffff640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf80137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.388 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf7f7fe640 1 -- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbf80098560 msgr2=0x7fbf80137450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf7f7fe640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbf80098560 0x7fbf80137450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf7f7fe640 1 -- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 msgr2=0x7fbf80137990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf7f7fe640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf80137990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf7f7fe640 1 -- 172.21.15.177:0/2173579272 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78009450 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf7f7fe640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 0x7fbf80136f10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000c890 tx=0x7fbf70001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000ce80 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf70005d60 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf74009390 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000a870 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.383+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf801157d0 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.384+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fbf7000aca0 con 0x7fbf80097a80 2024-09-13T18:35:09.389 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.384+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8005ed70 con 0x7fbf80097a80 2024-09-13T18:35:09.390 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.385+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2455+0+0 (secure 0 0 0) 0x7fbf70062e30 con 0x7fbf80097a80 2024-09-13T18:35:09.390 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.388+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fbf7003e180 con 0x7fbf80097a80 2024-09-13T18:35:09.522 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.520+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf800672c0 con 0x7fbf80097a80 2024-09-13T18:35:09.523 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.521+0000 7fbf677fe640 1 -- 172.21.15.177:0/2173579272 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+6575 (secure 0 0 0) 0x7fbf70041990 con 0x7fbf80097a80 2024-09-13T18:35:09.523 INFO:teuthology.misc.health.smithi177.stdout: 2024-09-13T18:35:09.523 INFO:teuthology.misc.health.smithi177.stdout:{"epoch":13,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:35:07.633215+0000","last_up_change":"2024-09-13T18:35:03.616896+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":32756}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":32756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":32756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":32756}]},"public_addr":"172.21.15.177:6804/32756","cluster_addr":"172.21.15.177:6806/32756","heartbeat_back_addr":"172.21.15.177:6810/32756","heartbeat_front_addr":"172.21.15.177:6809/32756","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.522+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 >> v1:172.21.15.177:6816/32265 conn(0x7fbf6c04cb70 legacy=0x7fbf6c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.522+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 msgr2=0x7fbf80136f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.522+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 0x7fbf80136f10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000c890 tx=0x7fbf70001610 comp rx=0 tx=0).stop 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.522+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 shutdown_connections 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.522+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf8012d230 0x7fbf80137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.523+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbf80098560 0x7fbf80137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.523+0000 7fbf85b9d640 1 --2- 172.21.15.177:0/2173579272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80097a80 0x7fbf80136f10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.525 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.523+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 >> 172.21.15.177:0/2173579272 conn(0x7fbf8006baa0 msgr2=0x7fbf8012e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:09.526 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.523+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 shutdown_connections 2024-09-13T18:35:09.526 INFO:teuthology.misc.health.smithi177.stderr:2024-09-13T18:35:09.523+0000 7fbf85b9d640 1 -- 172.21.15.177:0/2173579272 wait complete. 2024-09-13T18:35:09.534 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-13T18:35:09.534 INFO:tasks.ceph:Creating RBD pool 2024-09-13T18:35:09.534 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T18:35:09.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 Processor -- start 2024-09-13T18:35:09.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 -- start start 2024-09-13T18:35:09.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd2ec08f230 0x7fd2ec08f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 0x7fd2ec12c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec12ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ec092200 con 0x7fd2ec08f230 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec095700 con 0x7fd2ec12c8a0 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2f1eef640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec090530 con 0x7fd2ec08fb40 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2eb7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd2ec08f230 0x7fd2ec08f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2eaffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 0x7fd2ec12c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2ebfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec12ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2eb7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd2ec08f230 0x7fd2ec08f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41412/0 (socket says 172.21.15.177:41412) 2024-09-13T18:35:09.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2eb7fe640 1 -- 172.21.15.177:0/3017200749 learned_addr learned my addr 172.21.15.177:0/3017200749 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.625+0000 7fd2eaffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 0x7fd2ec12c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51072/0 (socket says 172.21.15.177:51072) 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eb7fe640 1 -- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 msgr2=0x7fd2ec12c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eb7fe640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 0x7fd2ec12c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eb7fe640 1 -- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 msgr2=0x7fd2ec12ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eb7fe640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec12ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eb7fe640 1 -- 172.21.15.177:0/3017200749 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec090770 con 0x7fd2ec08f230 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eaffd640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 0x7fd2ec12c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:09.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2ebfff640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec12ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:09.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2eb7fe640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08f230 0x7fd2ec08f600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009970 tx=0x7fd2dc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b41278d58148daa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:09.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.626+0000 7fd2ea7fc640 1 -- 172.21.15.177:0/3017200749 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc02aca0 con 0x7fd2ec08f230 2024-09-13T18:35:09.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2ea7fc640 1 -- 172.21.15.177:0/3017200749 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc02ae00 con 0x7fd2ec08f230 2024-09-13T18:35:09.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2ea7fc640 1 -- 172.21.15.177:0/3017200749 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc032670 con 0x7fd2ec08f230 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 -- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08f230 msgr2=0x7fd2ec08f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08f230 0x7fd2ec08f600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009970 tx=0x7fd2dc0091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 -- 172.21.15.177:0/3017200749 shutdown_connections 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec12ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08fb40 0x7fd2ec12c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/3017200749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08f230 0x7fd2ec08f600 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 -- 172.21.15.177:0/3017200749 >> 172.21.15.177:0/3017200749 conn(0x7fd2ec09be30 msgr2=0x7fd2ec097ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 -- 172.21.15.177:0/3017200749 shutdown_connections 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.627+0000 7fd2f1eef640 1 -- 172.21.15.177:0/3017200749 wait complete. 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 Processor -- start 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 -- start start 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 0x7fd2ec08bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 0x7fd2ec08c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec08c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec120440 con 0x7fd2ec08fb40 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec12b850 con 0x7fd2ec12c8a0 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2f1eef640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec08d550 con 0x7fd2ec08f230 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.628+0000 7fd2eb7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 0x7fd2ec08bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 0x7fd2ec08c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 0x7fd2ec08c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41428/0 (socket says 172.21.15.177:41428) 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eb7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 0x7fd2ec08bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51074/0 (socket says 172.21.15.177:51074) 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 -- 172.21.15.177:0/1163095716 learned_addr learned my addr 172.21.15.177:0/1163095716 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2ebfff640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec08c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 -- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 msgr2=0x7fd2ec08bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 0x7fd2ec08bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 -- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 msgr2=0x7fd2ec08c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec08c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 -- 172.21.15.177:0/1163095716 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e0009390 con 0x7fd2ec08fb40 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eb7fe640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 0x7fd2ec08bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2ebfff640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec08c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2eaffd640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 0x7fd2ec08c100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000ecb0 tx=0x7fd2e00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000cb00 con 0x7fd2ec08fb40 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e000cc60 con 0x7fd2ec08fb40 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e4009a20 con 0x7fd2ec08fb40 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0013470 con 0x7fd2ec08fb40 2024-09-13T18:35:09.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.629+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc009450 con 0x7fd2ec08fb40 2024-09-13T18:35:09.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.630+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec05ed70 con 0x7fd2ec08fb40 2024-09-13T18:35:09.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.630+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fd2e00137e0 con 0x7fd2ec08fb40 2024-09-13T18:35:09.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.631+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2455+0+0 (secure 0 0 0) 0x7fd2e0063380 con 0x7fd2ec08fb40 2024-09-13T18:35:09.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.634+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fd2e003e650 con 0x7fd2ec08fb40 2024-09-13T18:35:09.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:09.765+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fd2ec0969b0 con 0x7fd2ec08fb40 2024-09-13T18:35:10.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.719+0000 7fd2d37fe640 1 -- 172.21.15.177:0/1163095716 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fd2e0041e60 con 0x7fd2ec08fb40 2024-09-13T18:35:10.722 INFO:teuthology.orchestra.run.smithi177.stderr:pool 'rbd' created 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 >> v1:172.21.15.177:6816/32265 conn(0x7fd2d804cb70 legacy=0x7fd2d804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 msgr2=0x7fd2ec08c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 0x7fd2ec08c100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000ecb0 tx=0x7fd2e00090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 shutdown_connections 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2ec12c8a0 0x7fd2ec08c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2ec08fb40 0x7fd2ec08c100 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000ecb0 tx=0x7fd2e00090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 --2- 172.21.15.177:0/1163095716 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd2ec08f230 0x7fd2ec08bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.723+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 >> 172.21.15.177:0/1163095716 conn(0x7fd2ec09be30 msgr2=0x7fd2ec094ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.724+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 shutdown_connections 2024-09-13T18:35:10.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.724+0000 7fd2f1eef640 1 -- 172.21.15.177:0/1163095716 wait complete. 2024-09-13T18:35:10.739 DEBUG:teuthology.orchestra.run.smithi177:> rbd --cluster ceph pool init rbd 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.784+0000 7fdeb4c2bb80 1 Processor -- start 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.784+0000 7fdeb4c2bb80 1 -- start start 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb4c2bb80 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b9209b0 0x55609b920d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb4c2bb80 1 --2- >> v2:172.21.15.177:3300/0 conn(0x55609b921ec0 0x55609b9212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb4c2bb80 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55609b84f0e0 con 0x55609b90ed20 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb4c2bb80 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x55609b888260 con 0x55609b921ec0 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb4c2bb80 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x55609b883f40 con 0x55609b9209b0 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb1f4c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x55609b921ec0 0x55609b9212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:10.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb1f4c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x55609b921ec0 0x55609b9212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41440/0 (socket says 172.21.15.177:41440) 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb2f4e640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x55609b90ed20 0x55609b90f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47130/0 (socket says 172.21.15.177:47130) 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb274d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b9209b0 0x55609b920d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/4287179015 learned_addr learned my addr 172.21.15.177:0/4287179015 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb1f4c640 1 -- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b9209b0 msgr2=0x55609b920d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb1f4c640 1 --2- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b9209b0 0x55609b920d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb1f4c640 1 -- 172.21.15.177:0/4287179015 >> v1:172.21.15.177:6789/0 conn(0x55609b90ed20 legacy=0x55609b90f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb1f4c640 1 -- 172.21.15.177:0/4287179015 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55609b828270 con 0x55609b921ec0 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.785+0000 7fdeb174b640 1 -- 172.21.15.177:0/4287179015 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946051917 0 0) 0x55609b84f0e0 con 0x55609b90ed20 2024-09-13T18:35:10.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.786+0000 7fdeb274d640 1 --2- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b9209b0 0x55609b920d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:10.791 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.787+0000 7fdeb1f4c640 1 --2- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b921ec0 0x55609b9212c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00a680 tx=0x7fde9c045cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7acae9a7557804 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:10.791 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.787+0000 7fdeb174b640 1 -- 172.21.15.177:0/4287179015 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c04d4e0 con 0x55609b921ec0 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb174b640 1 -- 172.21.15.177:0/4287179015 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c04d640 con 0x55609b921ec0 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb174b640 1 -- 172.21.15.177:0/4287179015 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c04d7d0 con 0x55609b921ec0 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b921ec0 msgr2=0x55609b9212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 --2- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b921ec0 0x55609b9212c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00a680 tx=0x7fde9c045cd0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/4287179015 shutdown_connections 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 --2- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b921ec0 0x55609b9212c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 --2- 172.21.15.177:0/4287179015 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b9209b0 0x55609b920d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/4287179015 >> 172.21.15.177:0/4287179015 conn(0x55609b92a9e0 msgr2=0x55609b92ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/4287179015 shutdown_connections 2024-09-13T18:35:10.792 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.788+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/4287179015 wait complete. 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 Processor -- start 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 -- start start 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b90ed20 0x55609b9c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b9209b0 0x55609b9c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55609b921ec0 0x55609b9c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609b846ba0 con 0x55609b9209b0 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55609b846840 con 0x55609b921ec0 2024-09-13T18:35:10.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb4c2bb80 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x55609b8097a0 con 0x55609b90ed20 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb274d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b90ed20 0x55609b9c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb1f4c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b9209b0 0x55609b9c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb274d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b90ed20 0x55609b9c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51086/0 (socket says 172.21.15.177:51086) 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb1f4c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b9209b0 0x55609b9c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41454/0 (socket says 172.21.15.177:41454) 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb274d640 1 -- 172.21.15.177:0/2408330385 learned_addr learned my addr 172.21.15.177:0/2408330385 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.789+0000 7fdeb2f4e640 1 --2- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55609b921ec0 0x55609b9c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb274d640 1 -- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55609b921ec0 msgr2=0x55609b9c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb274d640 1 --2- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55609b921ec0 0x55609b9c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb274d640 1 -- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b9209b0 msgr2=0x55609b9c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb274d640 1 --2- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b9209b0 0x55609b9c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb274d640 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea400a0e0 con 0x55609b90ed20 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb1f4c640 1 --2- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55609b9209b0 0x55609b9c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:10.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb274d640 1 --2- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b90ed20 0x55609b9c8c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdea400d930 tx=0x7fdea4007040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400a3e0 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400b940 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400bd00 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c005cf0 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.790+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55609b8350e0 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.791+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fdea4012660 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.791+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2797+0+0 (secure 0 0 0) 0x7fdea4062700 con 0x55609b90ed20 2024-09-13T18:35:10.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:10.792+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x55609b828600 con 0x55609b90ed20 2024-09-13T18:35:11.724 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.720+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdea403d6b0 con 0x55609b90ed20 2024-09-13T18:35:11.724 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.720+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x55609b846630 con 0x55609b90ed20 2024-09-13T18:35:11.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.730+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea403d6b0 con 0x55609b90ed20 2024-09-13T18:35:11.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.730+0000 7fdeb0f4a640 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7fde98002a10 con 0x55609b90ed20 2024-09-13T18:35:11.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.731+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 642+0+0 (secure 0 0 0) 0x7fdea4041430 con 0x55609b90ed20 2024-09-13T18:35:11.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.731+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> v1:172.21.15.177:6800/32755 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x55609b9d3a00 con 0x55609b9cff00 2024-09-13T18:35:11.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.733+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/2408330385 <== osd.3 v1:172.21.15.177:6800/32755 1 ==== osd_op_reply(1 rbd_trash [create] v15'1 uv1 ondisk = 0) v8 ==== 153+0+0 (unknown 2722885549 0 0) 0x7fdeac001be0 con 0x55609b9cff00 2024-09-13T18:35:11.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.735+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> v1:172.21.15.177:6800/32755 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x55609ba20f20 con 0x55609b9cff00 2024-09-13T18:35:11.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.736+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/2408330385 <== osd.3 v1:172.21.15.177:6800/32755 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 2354864950 0 0) 0x7fdeac001be0 con 0x55609b9cff00 2024-09-13T18:35:11.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.736+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 --> v1:172.21.15.177:6800/32755 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x55609ba20010 con 0x55609b9cff00 2024-09-13T18:35:11.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.736+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/2408330385 <== osd.3 v1:172.21.15.177:6800/32755 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 2366413114 0 0) 0x7fdeac001be0 con 0x55609b9cff00 2024-09-13T18:35:11.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:11.736+0000 7fdeb0f4a640 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7fde98003db0 con 0x55609b90ed20 2024-09-13T18:35:12.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:12.730+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 9 ==== pool_op_reply(tid 4 (0) Success v16) v1 ==== 55+0+0 (secure 0 0 0) 0x7fdea4041090 con 0x55609b90ed20 2024-09-13T18:35:12.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:12.730+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7fde94051f20 con 0x55609b90ed20 2024-09-13T18:35:12.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:12.730+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 10 ==== osd_map(16..16 src has 1..16) v4 ==== 618+0+0 (secure 0 0 0) 0x7fdea403dfb0 con 0x55609b90ed20 2024-09-13T18:35:12.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:12.731+0000 7fdeb0f4a640 1 -- 172.21.15.177:0/2408330385 --> v1:172.21.15.177:6800/32755 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7fde980056f0 con 0x55609b9cff00 2024-09-13T18:35:12.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:12.732+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/2408330385 <== osd.3 v1:172.21.15.177:6800/32755 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v16'1 uv1 ondisk = 0) v8 ==== 194+0+0 (unknown 1230306805 0 0) 0x7fdeac001be0 con 0x55609b9cff00 2024-09-13T18:35:12.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:12.732+0000 7fdeb0f4a640 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v16) v4 -- 0x7fde98005240 con 0x55609b90ed20 2024-09-13T18:35:13.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.734+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 11 ==== pool_op_reply(tid 6 (0) Success v17) v1 ==== 43+0+0 (secure 0 0 0) 0x7fdea4041270 con 0x55609b90ed20 2024-09-13T18:35:13.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.734+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7fde940520b0 con 0x55609b90ed20 2024-09-13T18:35:13.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.735+0000 7fdeaa7fc640 1 -- 172.21.15.177:0/2408330385 <== mon.2 v2:172.21.15.177:3302/0 12 ==== osd_map(17..17 src has 1..17) v4 ==== 646+0+0 (secure 0 0 0) 0x7fdea4061890 con 0x55609b90ed20 2024-09-13T18:35:13.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.735+0000 7fdeb0f4a640 1 -- 172.21.15.177:0/2408330385 --> v1:172.21.15.177:6800/32755 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x7fde98005f90 con 0x55609b9cff00 2024-09-13T18:35:13.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.741+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/2408330385 <== osd.3 v1:172.21.15.177:6800/32755 5 ==== osd_op_reply(7 rbd_info [write 0~19] v17'2 uv2 ondisk = 0) v8 ==== 152+0+0 (unknown 3144429724 0 0) 0x7fdeac001be0 con 0x55609b9cff00 2024-09-13T18:35:13.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.741+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 >> v1:172.21.15.177:6800/32755 conn(0x55609b9cff00 legacy=0x55609b9d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 >> v1:172.21.15.177:6816/32265 conn(0x7fde9404b4a0 legacy=0x7fde9404d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b90ed20 msgr2=0x55609b9c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 --2- 172.21.15.177:0/2408330385 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x55609b90ed20 0x55609b9c8c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdea400d930 tx=0x7fdea4007040 comp rx=0 tx=0).stop 2024-09-13T18:35:13.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb2f4e640 1 -- 172.21.15.177:0/2408330385 reap_dead start 2024-09-13T18:35:13.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 shutdown_connections 2024-09-13T18:35:13.746 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 >> 172.21.15.177:0/2408330385 conn(0x55609b92a9e0 msgr2=0x55609b919930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:13.746 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 shutdown_connections 2024-09-13T18:35:13.746 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.742+0000 7fdeb4c2bb80 1 -- 172.21.15.177:0/2408330385 wait complete. 2024-09-13T18:35:13.749 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T18:35:13.750 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T18:35:13.750 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T18:35:13.750 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T18:35:13.751 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 Processor -- start 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 -- start start 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 0x7fafb8095d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fafb80962a0 0x7fafb812e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb8062fa0 con 0x7fafb812e670 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb8063550 con 0x7fafb80962a0 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbff1d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafb806ebf0 con 0x7fafb8095990 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbdcb9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 0x7fafb8095d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbd4b8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fafb80962a0 0x7fafb812e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:13.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbdcb9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 0x7fafb8095d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51098/0 (socket says 172.21.15.177:51098) 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbdcb9640 1 -- 172.21.15.177:0/1242531481 learned_addr learned my addr 172.21.15.177:0/1242531481 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.847+0000 7fafbe4ba640 1 --1- 172.21.15.177:0/1242531481 >> v1:172.21.15.177:6789/0 conn(0x7fafb812e670 0x7fafb8130a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47140/0 (socket says 172.21.15.177:47140) 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbd4b8640 1 -- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 msgr2=0x7fafb8095d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbd4b8640 1 --2- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 0x7fafb8095d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbd4b8640 1 -- 172.21.15.177:0/1242531481 >> v1:172.21.15.177:6789/0 conn(0x7fafb812e670 legacy=0x7fafb8130a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbd4b8640 1 -- 172.21.15.177:0/1242531481 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb808e450 con 0x7fafb80962a0 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbccb7640 1 -- 172.21.15.177:0/1242531481 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103134051 0 0) 0x7fafb8062fa0 con 0x7fafb812e670 2024-09-13T18:35:13.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbdcb9640 1 --2- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 0x7fafb8095d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.848+0000 7fafbd4b8640 1 --2- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb812e130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fafac009850 tx=0x7fafac0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43846206c26ef53b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbccb7640 1 -- 172.21.15.177:0/1242531481 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac02ad00 con 0x7fafb80962a0 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbccb7640 1 -- 172.21.15.177:0/1242531481 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac02ae60 con 0x7fafb80962a0 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbccb7640 1 -- 172.21.15.177:0/1242531481 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac032670 con 0x7fafb80962a0 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 -- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 msgr2=0x7fafb812e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb812e130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fafac009850 tx=0x7fafac0090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 -- 172.21.15.177:0/1242531481 shutdown_connections 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb812e130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:13.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1242531481 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb8095990 0x7fafb8095d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 -- 172.21.15.177:0/1242531481 >> 172.21.15.177:0/1242531481 conn(0x7fafb8096a80 msgr2=0x7fafb8096e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 -- 172.21.15.177:0/1242531481 shutdown_connections 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.849+0000 7fafbff1d640 1 -- 172.21.15.177:0/1242531481 wait complete. 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 Processor -- start 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 -- start start 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 0x7fafb812d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb811f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb812e670 0x7fafb811f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:13.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8121d30 con 0x7fafb80962a0 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb808d8d0 con 0x7fafb8095990 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbff1d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafb812ded0 con 0x7fafb812e670 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbd4b8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb811f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbe4ba640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb812e670 0x7fafb811f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.850+0000 7fafbdcb9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 0x7fafb812d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbd4b8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb811f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41472/0 (socket says 172.21.15.177:41472) 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbd4b8640 1 -- 172.21.15.177:0/1035996761 learned_addr learned my addr 172.21.15.177:0/1035996761 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbdcb9640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 0x7fafb812d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41450/0 (socket says 172.21.15.177:41450) 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbe4ba640 1 -- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 msgr2=0x7fafb812d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbe4ba640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 0x7fafb812d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbe4ba640 1 -- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 msgr2=0x7fafb811f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:13.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbe4ba640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb811f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbe4ba640 1 -- 172.21.15.177:0/1035996761 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8009450 con 0x7fafb812e670 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbd4b8640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb811f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbdcb9640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 0x7fafb812d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbe4ba640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb812e670 0x7fafb811f5a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c860 tx=0x7fafb0001630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00016c0 con 0x7fafb812e670 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0005d60 con 0x7fafb812e670 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac009390 con 0x7fafb812e670 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000a830 con 0x7fafb812e670 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.851+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb81218f0 con 0x7fafb812e670 2024-09-13T18:35:13.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.852+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb805ed70 con 0x7fafb812e670 2024-09-13T18:35:13.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.853+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fafb000ac60 con 0x7fafb812e670 2024-09-13T18:35:13.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.853+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fafb0062e60 con 0x7fafb812e670 2024-09-13T18:35:13.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:13.857+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fafb003e0e0 con 0x7fafb812e670 2024-09-13T18:35:14.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.013+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fafb8089690 con 0x7fafb812e670 2024-09-13T18:35:14.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.015+0000 7fafb5ffb640 1 -- 172.21.15.177:0/1035996761 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+116186 (secure 0 0 0) 0x7fafb00418f0 con 0x7fafb812e670 2024-09-13T18:35:14.020 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:14.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.017+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 >> v1:172.21.15.177:6816/32265 conn(0x7fafa404cb70 legacy=0x7fafa404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.017+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb812e670 msgr2=0x7fafb811f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.017+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb812e670 0x7fafb811f5a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c860 tx=0x7fafb0001630 comp rx=0 tx=0).stop 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 shutdown_connections 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fafb812e670 0x7fafb811f5a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafb80962a0 0x7fafb811f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 --2- 172.21.15.177:0/1035996761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafb8095990 0x7fafb812d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 >> 172.21.15.177:0/1035996761 conn(0x7fafb8096a80 msgr2=0x7fafb80950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 shutdown_connections 2024-09-13T18:35:14.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.018+0000 7fafbff1d640 1 -- 172.21.15.177:0/1035996761 wait complete. 2024-09-13T18:35:14.031 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6816","nonce":32265}]},"active_addr":"172.21.15.177:6816/32265","active_change":"2024-09-13T18:34:56.604440+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3625374812}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2735606740}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":1782828617}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2408340411}]}]} 2024-09-13T18:35:14.035 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T18:35:14.035 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T18:35:14.036 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:35:14.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 Processor -- start 2024-09-13T18:35:14.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 -- start start 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0094d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc00688c0 0x7f7bc0132220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc0097920 con 0x7f7bc0095290 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0069780 con 0x7f7bc00688c0 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bc4add640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0092760 con 0x7f7bc0094980 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bbe59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0094d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bbed9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc00688c0 0x7f7bc0132220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bbdd9b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7bc0095290 0x7f7bc00681b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47150/0 (socket says 172.21.15.177:47150) 2024-09-13T18:35:14.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.126+0000 7f7bbdd9b640 1 -- 172.21.15.177:0/44226831 learned_addr learned my addr 172.21.15.177:0/44226831 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:14.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.127+0000 7f7bbd59a640 1 -- 172.21.15.177:0/44226831 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92772978 0 0) 0x7f7bc0097920 con 0x7f7bc0095290 2024-09-13T18:35:14.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.127+0000 7f7bbe59c640 1 -- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc00688c0 msgr2=0x7f7bc0132220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.127+0000 7f7bbe59c640 1 --2- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc00688c0 0x7f7bc0132220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.127+0000 7f7bbe59c640 1 -- 172.21.15.177:0/44226831 >> v1:172.21.15.177:6789/0 conn(0x7f7bc0095290 legacy=0x7f7bc00681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.127+0000 7f7bbe59c640 1 -- 172.21.15.177:0/44226831 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc006a1a0 con 0x7f7bc0094980 2024-09-13T18:35:14.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.127+0000 7f7bbed9d640 1 --2- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc00688c0 0x7f7bc0132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bbe59c640 1 --2- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0094d50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00098b0 tx=0x7f7bb00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8548f1bcdb0207c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bbd59a640 1 -- 172.21.15.177:0/44226831 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb002ac40 con 0x7f7bc0094980 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bbd59a640 1 -- 172.21.15.177:0/44226831 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb002ada0 con 0x7f7bc0094980 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bbd59a640 1 -- 172.21.15.177:0/44226831 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0032670 con 0x7f7bc0094980 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 -- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 msgr2=0x7f7bc0094d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 --2- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0094d50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00098b0 tx=0x7f7bb00090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 -- 172.21.15.177:0/44226831 shutdown_connections 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 --2- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc00688c0 0x7f7bc0132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 --2- 172.21.15.177:0/44226831 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0094d50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 -- 172.21.15.177:0/44226831 >> 172.21.15.177:0/44226831 conn(0x7f7bc0097b50 msgr2=0x7f7bc0097f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 -- 172.21.15.177:0/44226831 shutdown_connections 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.128+0000 7f7bc4add640 1 -- 172.21.15.177:0/44226831 wait complete. 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 Processor -- start 2024-09-13T18:35:14.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 -- start start 2024-09-13T18:35:14.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 0x7f7bc01393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc0095290 0x7f7bc0139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0122a80 con 0x7f7bc00688c0 2024-09-13T18:35:14.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0121380 con 0x7f7bc0095290 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bc4add640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0090720 con 0x7f7bc0094980 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bbe59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 0x7f7bc01393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bbdd9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bbe59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 0x7f7bc01393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41474/0 (socket says 172.21.15.177:41474) 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.129+0000 7f7bbe59c640 1 -- 172.21.15.177:0/2230371472 learned_addr learned my addr 172.21.15.177:0/2230371472 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc0095290 0x7f7bc0139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 -- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 msgr2=0x7f7bc0139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 -- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 msgr2=0x7f7bc01393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 0x7f7bc01393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 -- 172.21.15.177:0/2230371472 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8009a20 con 0x7f7bc0095290 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbdd9b640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbe59c640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 0x7f7bc01393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bbed9d640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc0095290 0x7f7bc0139e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009f10 tx=0x7f7bb800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800cdb0 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb800f040 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb0009390 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80146d0 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.130+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc01202b0 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.131+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc0091010 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.131+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f7bb8003020 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.132+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f7bb8063bc0 con 0x7f7bc0095290 2024-09-13T18:35:14.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.135+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f7bb803ed00 con 0x7f7bc0095290 2024-09-13T18:35:14.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.265+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bc005ed70 con 0x7f7bc0095290 2024-09-13T18:35:14.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.266+0000 7f7bae7fc640 1 -- 172.21.15.177:0/2230371472 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8201 (secure 0 0 0) 0x7f7bb8018070 con 0x7f7bc0095290 2024-09-13T18:35:14.269 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:14.269 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":17,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:35:13.728586+0000","last_up_change":"2024-09-13T18:35:03.616896+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":32756}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":32756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":32756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":32756}]},"public_addr":"172.21.15.177:6804/32756","cluster_addr":"172.21.15.177:6806/32756","heartbeat_back_addr":"172.21.15.177:6810/32756","heartbeat_front_addr":"172.21.15.177:6809/32756","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:35:14.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.267+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 >> v1:172.21.15.177:6816/32265 conn(0x7f7ba004cb70 legacy=0x7f7ba004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.267+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc0095290 msgr2=0x7f7bc0139e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.267+0000 7f7bc4add640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc0095290 0x7f7bc0139e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009f10 tx=0x7f7bb800b040 comp rx=0 tx=0).stop 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 shutdown_connections 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc0095290 0x7f7bc0139e50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7bc0094980 0x7f7bc0139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 --2- 172.21.15.177:0/2230371472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc00688c0 0x7f7bc01393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 >> 172.21.15.177:0/2230371472 conn(0x7f7bc0097b50 msgr2=0x7f7bc0099db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 shutdown_connections 2024-09-13T18:35:14.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.268+0000 7f7bc4add640 1 -- 172.21.15.177:0/2230371472 wait complete. 2024-09-13T18:35:14.281 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T18:35:14.282 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:35:14.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.372+0000 7fc31cf5d640 1 Processor -- start 2024-09-13T18:35:14.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.372+0000 7fc31cf5d640 1 -- start start 2024-09-13T18:35:14.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.372+0000 7fc31cf5d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc318094980 0x7fc318094d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.372+0000 7fc31cf5d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318095290 0x7fc3180681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31cf5d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc318097920 con 0x7fc3180686f0 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31cf5d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc318069780 con 0x7fc318095290 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31cf5d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc318092760 con 0x7fc318094980 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc316d9d640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc3180686f0 0x7fc318131210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47158/0 (socket says 172.21.15.177:47158) 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc30dd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318095290 0x7fc3180681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc316d9d640 1 -- 172.21.15.177:0/503422769 learned_addr learned my addr 172.21.15.177:0/503422769 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31659c640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc318094980 0x7fc318094d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31659c640 1 -- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318095290 msgr2=0x7fc3180681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc315d9b640 1 -- 172.21.15.177:0/503422769 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792213240 0 0) 0x7fc318097920 con 0x7fc3180686f0 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31659c640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318095290 0x7fc3180681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31659c640 1 -- 172.21.15.177:0/503422769 >> v1:172.21.15.177:6789/0 conn(0x7fc3180686f0 legacy=0x7fc318131210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc31659c640 1 -- 172.21.15.177:0/503422769 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31806a1a0 con 0x7fc318094980 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.373+0000 7fc30dd9b640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318095290 0x7fc3180681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31659c640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc318094980 0x7fc318094d50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc308009880 tx=0x7fc3080090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29434717dad440ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc315d9b640 1 -- 172.21.15.177:0/503422769 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30802ac50 con 0x7fc318094980 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc315d9b640 1 -- 172.21.15.177:0/503422769 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30802adb0 con 0x7fc318094980 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc315d9b640 1 -- 172.21.15.177:0/503422769 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308032670 con 0x7fc318094980 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 -- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc318094980 msgr2=0x7fc318094d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc318094980 0x7fc318094d50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc308009880 tx=0x7fc3080090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 -- 172.21.15.177:0/503422769 shutdown_connections 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318095290 0x7fc3180681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/503422769 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc318094980 0x7fc318094d50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 -- 172.21.15.177:0/503422769 >> 172.21.15.177:0/503422769 conn(0x7fc318097b50 msgr2=0x7fc318097f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.380 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 -- 172.21.15.177:0/503422769 shutdown_connections 2024-09-13T18:35:14.381 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.374+0000 7fc31cf5d640 1 -- 172.21.15.177:0/503422769 wait complete. 2024-09-13T18:35:14.381 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.375+0000 7fc31cf5d640 1 Processor -- start 2024-09-13T18:35:14.381 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.375+0000 7fc31cf5d640 1 -- start start 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31cf5d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 0x7fc3181391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31cf5d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318094980 0x7fc318139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31cf5d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095290 0x7fc318139c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31cf5d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318120130 con 0x7fc318095290 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31cf5d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc318122440 con 0x7fc318094980 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31cf5d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc318122810 con 0x7fc3180686f0 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31659c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 0x7fc3181391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318094980 0x7fc318139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31659c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 0x7fc3181391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51138/0 (socket says 172.21.15.177:51138) 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31659c640 1 -- 172.21.15.177:0/652287032 learned_addr learned my addr 172.21.15.177:0/652287032 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.383 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc316d9d640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095290 0x7fc318139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 -- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 msgr2=0x7fc3181391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 0x7fc3181391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 -- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095290 msgr2=0x7fc318139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095290 0x7fc318139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 -- 172.21.15.177:0/652287032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300009390 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc31659c640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 0x7fc3181391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30dd9b640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318094980 0x7fc318139710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc30000ea20 tx=0x7fc3000090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300009670 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.376+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000ca10 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.377+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000cdd0 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.377+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc308009390 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.377+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc318090700 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.377+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc318091000 con 0x7fc318094980 2024-09-13T18:35:14.384 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.378+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fc300013680 con 0x7fc318094980 2024-09-13T18:35:14.385 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.378+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fc3000634c0 con 0x7fc318094980 2024-09-13T18:35:14.385 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.381+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fc30003e600 con 0x7fc318094980 2024-09-13T18:35:14.515 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.512+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc31805ed70 con 0x7fc318094980 2024-09-13T18:35:14.515 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.512+0000 7fc30effd640 1 -- 172.21.15.177:0/652287032 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8201 (secure 0 0 0) 0x7fc300041e10 con 0x7fc318094980 2024-09-13T18:35:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:14.516 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":17,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:35:13.728586+0000","last_up_change":"2024-09-13T18:35:03.616896+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":32756}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":32756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":32756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":32756}]},"public_addr":"172.21.15.177:6804/32756","cluster_addr":"172.21.15.177:6806/32756","heartbeat_back_addr":"172.21.15.177:6810/32756","heartbeat_front_addr":"172.21.15.177:6809/32756","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 >> v1:172.21.15.177:6816/32265 conn(0x7fc2f804cba0 legacy=0x7fc2f804f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318094980 msgr2=0x7fc318139710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318094980 0x7fc318139710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc30000ea20 tx=0x7fc3000090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 shutdown_connections 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095290 0x7fc318139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc318094980 0x7fc318139710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 --2- 172.21.15.177:0/652287032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc3180686f0 0x7fc3181391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 >> 172.21.15.177:0/652287032 conn(0x7fc318097b50 msgr2=0x7fc318092940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 shutdown_connections 2024-09-13T18:35:14.518 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.514+0000 7fc31cf5d640 1 -- 172.21.15.177:0/652287032 wait complete. 2024-09-13T18:35:14.528 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T18:35:14.529 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T18:35:14.530 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T18:35:14.530 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T18:35:14.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 Processor -- start 2024-09-13T18:35:14.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f4270d1d640 1 Processor -- start 2024-09-13T18:35:14.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f4270d1d640 1 -- start start 2024-09-13T18:35:14.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f494560f640 1 Processor -- start 2024-09-13T18:35:14.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 -- start start 2024-09-13T18:35:14.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f494560f640 1 -- start start 2024-09-13T18:35:14.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f4c4e82b640 1 Processor -- start 2024-09-13T18:35:14.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f4c4e82b640 1 -- start start 2024-09-13T18:35:14.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c4c0924d0 0x7f2c4c0928a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c12c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c093730 con 0x7f2c4c12c6a0 2024-09-13T18:35:14.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f4270d1d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f426c068260 0x7f426c068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c093d10 con 0x7f2c4c0924d0 2024-09-13T18:35:14.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f494560f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940098630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f2c5366f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c094830 con 0x7f2c4c092de0 2024-09-13T18:35:14.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.627+0000 7f4270d1d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c12fad0 0x7f426c131eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f493effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940098630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4270d1d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c069370 con 0x7f426c068b70 2024-09-13T18:35:14.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f493effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940098630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41496/0 (socket says 172.21.15.177:41496) 2024-09-13T18:35:14.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c50c0a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c12c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c4c0924d0 0x7f2c4c0928a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c4c0924d0 0x7f2c4c0928a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41488/0 (socket says 172.21.15.177:41488) 2024-09-13T18:35:14.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 -- 172.21.15.177:0/2178846497 learned_addr learned my addr 172.21.15.177:0/2178846497 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c50c0a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c12c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51146/0 (socket says 172.21.15.177:51146) 2024-09-13T18:35:14.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f493effd640 1 -- 172.21.15.177:0/2678578555 learned_addr learned my addr 172.21.15.177:0/2678578555 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f494560f640 1 --2- 172.21.15.177:0/2678578555 >> v2:172.21.15.177:3300/0 conn(0x7f494006e870 0x7f4940098b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f494560f640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 0x7f4940069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4e82b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c480924d0 0x7f4c480928a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4e82b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4c48092de0 0x7f4c4812c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4e82b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c4812c6a0 0x7f4c4812ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4e82b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c48093d10 con 0x7f4c48092de0 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4e82b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48093730 con 0x7f4c4812c6a0 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4e82b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48094830 con 0x7f4c480924d0 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4c48092de0 0x7f4c4812c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4c48092de0 0x7f4c4812c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41502/0 (socket says 172.21.15.177:41502) 2024-09-13T18:35:14.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 -- 172.21.15.177:0/37123513 learned_addr learned my addr 172.21.15.177:0/37123513 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4269d9b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f426c068b70 0x7f426c12e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47166/0 (socket says 172.21.15.177:47166) 2024-09-13T18:35:14.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4269d9b640 1 -- 172.21.15.177:0/942672294 learned_addr learned my addr 172.21.15.177:0/942672294 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:14.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f426a59c640 1 --2- 172.21.15.177:0/942672294 >> v2:172.21.15.177:3300/0 conn(0x7f426c068260 0x7f426c068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c069da0 con 0x7f426c068260 2024-09-13T18:35:14.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0994e0 con 0x7f426c12fad0 2024-09-13T18:35:14.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c47fff640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c480924d0 0x7f4c480928a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494006ba70 con 0x7f494006e870 2024-09-13T18:35:14.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494006ec40 con 0x7f4940064080 2024-09-13T18:35:14.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c4cdc8640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c4812c6a0 0x7f4c4812ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f426ad9d640 1 --2- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c12fad0 0x7f426c131eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f494006eea0 con 0x7f49400990b0 2024-09-13T18:35:14.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 -- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 msgr2=0x7f2c4c12c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 --2- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c12c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 -- 172.21.15.177:0/2178846497 >> v1:172.21.15.177:6789/0 conn(0x7f2c4c12c6a0 legacy=0x7f2c4c12ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f2c5140b640 1 -- 172.21.15.177:0/2178846497 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c094570 con 0x7f2c4c0924d0 2024-09-13T18:35:14.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f493f7fe640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 0x7f4940069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 -- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c480924d0 msgr2=0x7f4c480928a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c480924d0 0x7f4c480928a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 -- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c4812c6a0 msgr2=0x7f4c4812ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c4812c6a0 0x7f4c4812ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.628+0000 7f4c477fe640 1 -- 172.21.15.177:0/37123513 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48094570 con 0x7f4c48092de0 2024-09-13T18:35:14.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493e7fc640 1 --2- 172.21.15.177:0/2678578555 >> v2:172.21.15.177:3300/0 conn(0x7f494006e870 0x7f4940098b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f426a59c640 1 -- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c12fad0 msgr2=0x7f426c131eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f426a59c640 1 --2- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c12fad0 0x7f426c131eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f426a59c640 1 -- 172.21.15.177:0/942672294 >> v1:172.21.15.177:6789/0 conn(0x7f426c068b70 legacy=0x7f426c12e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493effd640 1 -- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 msgr2=0x7f4940069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493effd640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 0x7f4940069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493effd640 1 -- 172.21.15.177:0/2678578555 >> v2:172.21.15.177:3300/0 conn(0x7f494006e870 msgr2=0x7f4940098b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f426959a640 1 -- 172.21.15.177:0/942672294 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263983684 0 0) 0x7f426c069370 con 0x7f426c068b70 2024-09-13T18:35:14.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f426a59c640 1 -- 172.21.15.177:0/942672294 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0940f0 con 0x7f426c068260 2024-09-13T18:35:14.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493effd640 1 --2- 172.21.15.177:0/2678578555 >> v2:172.21.15.177:3300/0 conn(0x7f494006e870 0x7f4940098b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493effd640 1 -- 172.21.15.177:0/2678578555 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400970d0 con 0x7f4940064080 2024-09-13T18:35:14.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f493e7fc640 1 --2- 172.21.15.177:0/2678578555 >> v2:172.21.15.177:3300/0 conn(0x7f494006e870 0x7f4940098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:35:14.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f2c5140b640 1 --2- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c0928a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009910 tx=0x7f2c480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9100b3fac64fd7d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f4c477fe640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4812c160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009910 tx=0x7f4c3c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4d964940577cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f2c43fff640 1 -- 172.21.15.177:0/2178846497 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4802ad20 con 0x7f2c4c0924d0 2024-09-13T18:35:14.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f2c43fff640 1 -- 172.21.15.177:0/2178846497 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4802ae80 con 0x7f2c4c0924d0 2024-09-13T18:35:14.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.629+0000 7f2c43fff640 1 -- 172.21.15.177:0/2178846497 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48032670 con 0x7f2c4c0924d0 2024-09-13T18:35:14.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f4c46ffd640 1 -- 172.21.15.177:0/37123513 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c02ac80 con 0x7f4c48092de0 2024-09-13T18:35:14.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f4c46ffd640 1 -- 172.21.15.177:0/37123513 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c02ade0 con 0x7f4c48092de0 2024-09-13T18:35:14.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f493effd640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940098630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4930009970 tx=0x7f49300091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a0a860009b8dad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f4c46ffd640 1 -- 172.21.15.177:0/37123513 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c032670 con 0x7f4c48092de0 2024-09-13T18:35:14.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f493dffb640 1 -- 172.21.15.177:0/2678578555 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493002aca0 con 0x7f4940064080 2024-09-13T18:35:14.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f426a59c640 1 --2- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c068260 0x7f426c068630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f425c009850 tx=0x7f425c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415526e07583265c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f493dffb640 1 -- 172.21.15.177:0/2678578555 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493002ae00 con 0x7f4940064080 2024-09-13T18:35:14.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f493dffb640 1 -- 172.21.15.177:0/2678578555 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930032670 con 0x7f4940064080 2024-09-13T18:35:14.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f426959a640 1 -- 172.21.15.177:0/942672294 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c02ac60 con 0x7f426c068260 2024-09-13T18:35:14.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f426959a640 1 -- 172.21.15.177:0/942672294 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c02adc0 con 0x7f426c068260 2024-09-13T18:35:14.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f426959a640 1 -- 172.21.15.177:0/942672294 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c032670 con 0x7f426c068260 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 -- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 msgr2=0x7f2c4c0928a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 --2- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c0928a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009910 tx=0x7f2c480091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 -- 172.21.15.177:0/2178846497 shutdown_connections 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 --2- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c12c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 --2- 172.21.15.177:0/2178846497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c0928a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 -- 172.21.15.177:0/2178846497 >> 172.21.15.177:0/2178846497 conn(0x7f2c4c097ae0 msgr2=0x7f2c4c097ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 -- 172.21.15.177:0/2178846497 shutdown_connections 2024-09-13T18:35:14.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.630+0000 7f2c5366f640 1 -- 172.21.15.177:0/2178846497 wait complete. 2024-09-13T18:35:14.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 -- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 msgr2=0x7f4c4812c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4812c160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009910 tx=0x7f4c3c0091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 -- 172.21.15.177:0/37123513 shutdown_connections 2024-09-13T18:35:14.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c4812c6a0 0x7f4c4812ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4812c160 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/37123513 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c480924d0 0x7f4c480928a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 -- 172.21.15.177:0/37123513 >> 172.21.15.177:0/37123513 conn(0x7f4c48097ae0 msgr2=0x7f4c48097ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 msgr2=0x7f4940098630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940098630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4930009970 tx=0x7f49300091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 -- 172.21.15.177:0/37123513 shutdown_connections 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4c4e82b640 1 -- 172.21.15.177:0/37123513 wait complete. 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 shutdown_connections 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 0x7f4940069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 --2- 172.21.15.177:0/2678578555 >> v2:172.21.15.177:3300/0 conn(0x7f494006e870 0x7f4940098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 --2- 172.21.15.177:0/2678578555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940098630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 >> 172.21.15.177:0/2678578555 conn(0x7f4940096210 msgr2=0x7f4940096610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c068260 msgr2=0x7f426c068630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 --2- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c068260 0x7f426c068630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f425c009850 tx=0x7f425c0090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 shutdown_connections 2024-09-13T18:35:14.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f494560f640 1 -- 172.21.15.177:0/2678578555 wait complete. 2024-09-13T18:35:14.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 shutdown_connections 2024-09-13T18:35:14.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 --2- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c12fad0 0x7f426c131eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 --2- 172.21.15.177:0/942672294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c068260 0x7f426c068630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 >> 172.21.15.177:0/942672294 conn(0x7f426c099d30 msgr2=0x7f426c09a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f2c5366f640 1 Processor -- start 2024-09-13T18:35:14.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 shutdown_connections 2024-09-13T18:35:14.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f2c5366f640 1 -- start start 2024-09-13T18:35:14.656 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.631+0000 7f4270d1d640 1 -- 172.21.15.177:0/942672294 wait complete. 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5366f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5366f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c13a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5140b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5366f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c4c12c6a0 0x7f2c4c13a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5366f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1224f0 con 0x7f2c4c0924d0 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c50c0a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c13a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c50c0a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c13a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51186/0 (socket says 172.21.15.177:51186) 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c50c0a640 1 -- 172.21.15.177:0/196350105 learned_addr learned my addr 172.21.15.177:0/196350105 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c121600 con 0x7f2c4c12c6a0 2024-09-13T18:35:14.657 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c090c30 con 0x7f2c4c092de0 2024-09-13T18:35:14.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f4c4e82b640 1 Processor -- start 2024-09-13T18:35:14.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f2c51c0c640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c4c12c6a0 0x7f2c4c13a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f4270d1d640 1 Processor -- start 2024-09-13T18:35:14.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f494560f640 1 Processor -- start 2024-09-13T18:35:14.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f4c4e82b640 1 -- start start 2024-09-13T18:35:14.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.632+0000 7f4270d1d640 1 -- start start 2024-09-13T18:35:14.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 -- start start 2024-09-13T18:35:14.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c50c0a640 1 -- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c4c12c6a0 msgr2=0x7f2c4c13a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c50c0a640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c4c12c6a0 0x7f2c4c13a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c50c0a640 1 -- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 msgr2=0x7f2c4c139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c50c0a640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c139b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c50c0a640 1 -- 172.21.15.177:0/196350105 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48009450 con 0x7f2c4c092de0 2024-09-13T18:35:14.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c5140b640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c4c0924d0 0x7f2c4c139b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c50c0a640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c13a080 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00ea20 tx=0x7f2c3c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c51c0c640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c4c12c6a0 0x7f2c4c13a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00ca10 con 0x7f2c4c092de0 2024-09-13T18:35:14.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940124c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c00cb70 con 0x7f2c4c092de0 2024-09-13T18:35:14.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f4270d1d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f426c068260 0x7f426c122fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c013470 con 0x7f2c4c092de0 2024-09-13T18:35:14.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c1221e0 con 0x7f2c4c092de0 2024-09-13T18:35:14.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f4270d1d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c068b70 0x7f426c123520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f4c4e82b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c480924d0 0x7f4c4808e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f4270d1d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c12fad0 0x7f426c123a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f426a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f426c068260 0x7f426c122fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c090360 con 0x7f2c4c092de0 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f494006e870 0x7f49401251b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 0x7f49401256f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940126f40 con 0x7f494006e870 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49401281b0 con 0x7f4940064080 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f494560f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4940128470 con 0x7f49400990b0 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f493effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940124c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f493effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940124c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41530/0 (socket says 172.21.15.177:41530) 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f493effd640 1 -- 172.21.15.177:0/3884079735 learned_addr learned my addr 172.21.15.177:0/3884079735 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f426a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f426c068260 0x7f426c122fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41546/0 (socket says 172.21.15.177:41546) 2024-09-13T18:35:14.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c480924d0 0x7f4c4808e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493e7fc640 1 --2- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f494006e870 0x7f49401251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c480924d0 0x7f4c4808e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41562/0 (socket says 172.21.15.177:41562) 2024-09-13T18:35:14.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.633+0000 7f426a59c640 1 -- 172.21.15.177:0/4125295574 learned_addr learned my addr 172.21.15.177:0/4125295574 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f426ad9d640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c12fad0 0x7f426c123a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 -- 172.21.15.177:0/431490903 learned_addr learned my addr 172.21.15.177:0/431490903 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c1267c0 con 0x7f426c12fad0 2024-09-13T18:35:14.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c124ac0 con 0x7f426c068260 2024-09-13T18:35:14.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493effd640 1 -- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 msgr2=0x7f49401256f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493effd640 1 --2- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f49400990b0 0x7f49401256f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493effd640 1 -- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f494006e870 msgr2=0x7f49401251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f426c1263f0 con 0x7f426c068b70 2024-09-13T18:35:14.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493effd640 1 --2- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f494006e870 0x7f49401251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493effd640 1 -- 172.21.15.177:0/3884079735 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930009450 con 0x7f4940064080 2024-09-13T18:35:14.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4808eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c4812c6a0 0x7f4c4808f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48091af0 con 0x7f4c48092de0 2024-09-13T18:35:14.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48091d90 con 0x7f4c480924d0 2024-09-13T18:35:14.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48090090 con 0x7f4c4812c6a0 2024-09-13T18:35:14.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f493effd640 1 --2- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940124c70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f493002a610 tx=0x7f4930032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4269d9b640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c068b70 0x7f426c123520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c477fe640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4808eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 -- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c4812c6a0 msgr2=0x7f4c4808f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4c4812c6a0 0x7f4c4808f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930032e20 con 0x7f4940064080 2024-09-13T18:35:14.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49300315b0 con 0x7f4940064080 2024-09-13T18:35:14.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930031970 con 0x7f4940064080 2024-09-13T18:35:14.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c4c05ed70 con 0x7f2c4c092de0 2024-09-13T18:35:14.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 -- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 msgr2=0x7f4c4808eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4808eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c47fff640 1 -- 172.21.15.177:0/431490903 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c009450 con 0x7f4c480924d0 2024-09-13T18:35:14.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.634+0000 7f4c477fe640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c48092de0 0x7f4c4808eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:14.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49401286c0 con 0x7f4940064080 2024-09-13T18:35:14.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4940128ef0 con 0x7f4940064080 2024-09-13T18:35:14.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4269d9b640 1 -- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f426c068260 msgr2=0x7f426c122fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4269d9b640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f426c068260 0x7f426c122fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4269d9b640 1 -- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c12fad0 msgr2=0x7f426c123a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4c47fff640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c480924d0 0x7f4c4808e600 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800d890 tx=0x7f4c38008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4269d9b640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f426c12fad0 0x7f426c123a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4269d9b640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c009390 con 0x7f426c068b70 2024-09-13T18:35:14.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f426a59c640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f426c068260 0x7f426c122fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:14.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4269d9b640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c068b70 0x7f426c123520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f426000c500 tx=0x7f4260007ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800de20 con 0x7f4c480924d0 2024-09-13T18:35:14.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3800b8d0 con 0x7f4c480924d0 2024-09-13T18:35:14.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800bc70 con 0x7f4c480924d0 2024-09-13T18:35:14.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4812bc80 con 0x7f4c480924d0 2024-09-13T18:35:14.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260005aa0 con 0x7f426c068b70 2024-09-13T18:35:14.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4260005c20 con 0x7f426c068b70 2024-09-13T18:35:14.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260007320 con 0x7f426c068b70 2024-09-13T18:35:14.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.635+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f494005ed70 con 0x7f4940064080 2024-09-13T18:35:14.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4808f5c0 con 0x7f4c480924d0 2024-09-13T18:35:14.676 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f2c3c013820 con 0x7f2c4c092de0 2024-09-13T18:35:14.676 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f2c3c063470 con 0x7f2c4c092de0 2024-09-13T18:35:14.676 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f4930031d20 con 0x7f4940064080 2024-09-13T18:35:14.676 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 --> v1:172.21.15.177:6804/32756 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c30057020 con 0x7f2c30053340 2024-09-13T18:35:14.676 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3c0659b0 con 0x7f2c4c092de0 2024-09-13T18:35:14.677 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c12f610 con 0x7f426c068b70 2024-09-13T18:35:14.677 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.636+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c125f60 con 0x7f426c068b70 2024-09-13T18:35:14.677 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.637+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f493008b980 con 0x7f4940064080 2024-09-13T18:35:14.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.637+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 --> v1:172.21.15.177:6800/32755 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f492c056fa0 con 0x7f492c0532c0 2024-09-13T18:35:14.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.637+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4930066b40 con 0x7f4940064080 2024-09-13T18:35:14.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.637+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f4c38014640 con 0x7f4c480924d0 2024-09-13T18:35:14.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.637+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f4c38063800 con 0x7f4c480924d0 2024-09-13T18:35:14.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.637+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f426c05ed70 con 0x7f426c068b70 2024-09-13T18:35:14.679 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.638+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== osd.0 v1:172.21.15.177:6804/32756 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f2c30057020 con 0x7f2c30053340 2024-09-13T18:35:14.679 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.638+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> v1:172.21.15.177:6812/32757 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c4806a3c0 con 0x7f4c4808f810 2024-09-13T18:35:14.679 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.638+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f426001b3b0 con 0x7f426c068b70 2024-09-13T18:35:14.679 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.638+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== osd.3 v1:172.21.15.177:6800/32755 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f492c056fa0 con 0x7f492c0532c0 2024-09-13T18:35:14.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.639+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== osd.1 v1:172.21.15.177:6812/32757 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f4c4806a3c0 con 0x7f4c4808f810 2024-09-13T18:35:14.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.639+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f4260062700 con 0x7f426c068b70 2024-09-13T18:35:14.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.639+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 --> v1:172.21.15.177:6805/32758 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f424c056fa0 con 0x7f424c0532c0 2024-09-13T18:35:14.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.639+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4260064c60 con 0x7f426c068b70 2024-09-13T18:35:14.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.640+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== osd.2 v1:172.21.15.177:6805/32758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f424c056fa0 con 0x7f424c0532c0 2024-09-13T18:35:14.681 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 --> v1:172.21.15.177:6804/32756 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c4c120380 con 0x7f2c30053340 2024-09-13T18:35:14.681 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c417fa640 1 -- 172.21.15.177:0/196350105 <== osd.0 v1:172.21.15.177:6804/32756 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f2c4c120380 con 0x7f2c30053340 2024-09-13T18:35:14.681 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 --> v1:172.21.15.177:6800/32755 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f494006ddd0 con 0x7f492c0532c0 2024-09-13T18:35:14.681 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 >> v1:172.21.15.177:6804/32756 conn(0x7f2c30053340 legacy=0x7f2c300557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f4926ffd640 1 -- 172.21.15.177:0/3884079735 <== osd.3 v1:172.21.15.177:6800/32755 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f494006ddd0 con 0x7f492c0532c0 2024-09-13T18:35:14.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 >> v1:172.21.15.177:6816/32265 conn(0x7f2c3004ccc0 legacy=0x7f2c3004f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 msgr2=0x7f2c4c13a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c5366f640 1 --2- 172.21.15.177:0/196350105 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c4c092de0 0x7f2c4c13a080 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00ea20 tx=0x7f2c3c0090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.651+0000 7f2c51c0c640 1 -- 172.21.15.177:0/196350105 reap_dead start 2024-09-13T18:35:14.683 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 shutdown_connections 2024-09-13T18:35:14.683 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 >> 172.21.15.177:0/196350105 conn(0x7f2c4c097ae0 msgr2=0x7f2c4c0917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.683 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 shutdown_connections 2024-09-13T18:35:14.683 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f2c5366f640 1 -- 172.21.15.177:0/196350105 wait complete. 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 >> v1:172.21.15.177:6800/32755 conn(0x7f492c0532c0 legacy=0x7f492c055730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 >> v1:172.21.15.177:6816/32265 conn(0x7f492c04cc40 legacy=0x7f492c04f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 msgr2=0x7f4940124c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 --2- 172.21.15.177:0/3884079735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4940064080 0x7f4940124c70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f493002a610 tx=0x7f4930032000 comp rx=0 tx=0).stop 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f493f7fe640 1 -- 172.21.15.177:0/3884079735 reap_dead start 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 shutdown_connections 2024-09-13T18:35:14.684 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 >> 172.21.15.177:0/3884079735 conn(0x7f4940096210 msgr2=0x7f4940097620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.685 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 shutdown_connections 2024-09-13T18:35:14.685 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.652+0000 7f494560f640 1 -- 172.21.15.177:0/3884079735 wait complete. 2024-09-13T18:35:14.685 INFO:teuthology.orchestra.run.smithi177.stdout:38654705669 2024-09-13T18:35:14.686 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:35:14.686 INFO:teuthology.orchestra.run.smithi177.stdout:38654705669 2024-09-13T18:35:14.687 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:35:14.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.666+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 --> v1:172.21.15.177:6812/32757 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c4805ea60 con 0x7f4c4808f810 2024-09-13T18:35:14.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c2ffff640 1 -- 172.21.15.177:0/431490903 <== osd.1 v1:172.21.15.177:6812/32757 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f4c4805ea60 con 0x7f4c4808f810 2024-09-13T18:35:14.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 >> v1:172.21.15.177:6812/32757 conn(0x7f4c4808f810 legacy=0x7f4c4805e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 >> v1:172.21.15.177:6816/32265 conn(0x7f4c3404cc40 legacy=0x7f4c3404f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c480924d0 msgr2=0x7f4c4808e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 --2- 172.21.15.177:0/431490903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c480924d0 0x7f4c4808e600 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800d890 tx=0x7f4c38008040 comp rx=0 tx=0).stop 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4cdc8640 1 -- 172.21.15.177:0/431490903 reap_dead start 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 shutdown_connections 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 >> 172.21.15.177:0/431490903 conn(0x7f4c48097ae0 msgr2=0x7f4c4809a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 shutdown_connections 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4c4e82b640 1 -- 172.21.15.177:0/431490903 wait complete. 2024-09-13T18:35:14.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.667+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 --> v1:172.21.15.177:6805/32758 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f426c1243b0 con 0x7f424c0532c0 2024-09-13T18:35:14.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f425a7fc640 1 -- 172.21.15.177:0/4125295574 <== osd.2 v1:172.21.15.177:6805/32758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f426c1243b0 con 0x7f424c0532c0 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 >> v1:172.21.15.177:6805/32758 conn(0x7f424c0532c0 legacy=0x7f424c055730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 >> v1:172.21.15.177:6816/32265 conn(0x7f424c04cc40 legacy=0x7f424c04f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c068b70 msgr2=0x7f426c123520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 --2- 172.21.15.177:0/4125295574 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f426c068b70 0x7f426c123520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f426000c500 tx=0x7f4260007ca0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f426ad9d640 1 -- 172.21.15.177:0/4125295574 reap_dead start 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 shutdown_connections 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 >> 172.21.15.177:0/4125295574 conn(0x7f426c099d30 msgr2=0x7f426c092b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 shutdown_connections 2024-09-13T18:35:14.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.668+0000 7f4270d1d640 1 -- 172.21.15.177:0/4125295574 wait complete. 2024-09-13T18:35:14.692 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:14.692 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:35:14.693 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:14.693 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:35:14.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.823+0000 7f2fba24f640 1 Processor -- start 2024-09-13T18:35:14.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.823+0000 7f2fba24f640 1 -- start start 2024-09-13T18:35:14.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.823+0000 7f208451d640 1 Processor -- start 2024-09-13T18:35:14.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f133791d640 1 Processor -- start 2024-09-13T18:35:14.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 Processor -- start 2024-09-13T18:35:14.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f208451d640 1 -- start start 2024-09-13T18:35:14.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 -- start start 2024-09-13T18:35:14.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f133791d640 1 -- start start 2024-09-13T18:35:14.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fba24f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb4068260 0x7f2fb4068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fba24f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb412fad0 0x7f2fb4131eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fba24f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb4069370 con 0x7f2fb4068b70 2024-09-13T18:35:14.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fba24f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4069da0 con 0x7f2fb412fad0 2024-09-13T18:35:14.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fba24f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40994e0 con 0x7f2fb4068260 2024-09-13T18:35:14.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f58fc068260 0x7f58fc068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fb2ffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2fb4068b70 0x7f2fb412e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47168/0 (socket says 172.21.15.177:47168) 2024-09-13T18:35:14.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb4068260 0x7f2fb4068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fb2ffd640 1 -- 172.21.15.177:0/1348833051 learned_addr learned my addr 172.21.15.177:0/1348833051 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:14.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068b70 0x7f58fc12e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2fb3fff640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb412fad0 0x7f2fb4131eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f208451d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f207c068120 0x7f207c091450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58fc069370 con 0x7f58fc12e8f0 2024-09-13T18:35:14.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58fc069da0 con 0x7f58fc068260 2024-09-13T18:35:14.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f590148b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0994e0 con 0x7f58fc068b70 2024-09-13T18:35:14.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068b70 0x7f58fc12e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068b70 0x7f58fc12e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41584/0 (socket says 172.21.15.177:41584) 2024-09-13T18:35:14.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58f3fff640 1 -- 172.21.15.177:0/2053083300 learned_addr learned my addr 172.21.15.177:0/2053083300 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f208451d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 0x7f207c091d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f208451d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c12b0e0 0x7f207c12d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.824+0000 7f2081ab8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 0x7f207c091d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2081ab8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 0x7f207c091d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41578/0 (socket says 172.21.15.177:41578) 2024-09-13T18:35:14.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f20822b9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f207c068120 0x7f207c091450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f20822b9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f207c068120 0x7f207c091450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41566/0 (socket says 172.21.15.177:41566) 2024-09-13T18:35:14.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f20822b9640 1 -- 172.21.15.177:0/3688157889 learned_addr learned my addr 172.21.15.177:0/3688157889 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f133791d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f133006a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f133791d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330126cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f133791d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133006b8d0 con 0x7f1330064080 2024-09-13T18:35:14.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f133791d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133006eab0 con 0x7f1330126870 2024-09-13T18:35:14.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f133791d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1330099050 con 0x7f1330064450 2024-09-13T18:35:14.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb412fad0 msgr2=0x7f2fb4131eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c069570 con 0x7f207c068120 2024-09-13T18:35:14.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0927e0 con 0x7f207c091990 2024-09-13T18:35:14.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0922a0 con 0x7f207c12b0e0 2024-09-13T18:35:14.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f133006a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f133006a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51242/0 (socket says 172.21.15.177:51242) 2024-09-13T18:35:14.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 -- 172.21.15.177:0/2939803537 learned_addr learned my addr 172.21.15.177:0/2939803537 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2fb37fe640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb412fad0 0x7f2fb4131eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2fb27fc640 1 -- 172.21.15.177:0/1348833051 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841284751 0 0) 0x7f2fb4069370 con 0x7f2fb4068b70 2024-09-13T18:35:14.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2082aba640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c12b0e0 0x7f207c12d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1348833051 >> v1:172.21.15.177:6789/0 conn(0x7f2fb4068b70 legacy=0x7f2fb412e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1348833051 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40940f0 con 0x7f2fb4068260 2024-09-13T18:35:14.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58faffd640 1 --2- 172.21.15.177:0/2053083300 >> v2:172.21.15.177:3300/0 conn(0x7f58fc068260 0x7f58fc068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58fa7fc640 1 -- 172.21.15.177:0/2053083300 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120033204 0 0) 0x7f58fc069370 con 0x7f58fc12e8f0 2024-09-13T18:35:14.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58fa7fc640 1 -- 172.21.15.177:0/2053083300 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e80034e0 con 0x7f58fc12e8f0 2024-09-13T18:35:14.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58faffd640 1 -- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068b70 msgr2=0x7f58fc12e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58faffd640 1 --2- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068b70 0x7f58fc12e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58faffd640 1 -- 172.21.15.177:0/2053083300 >> v1:172.21.15.177:6789/0 conn(0x7f58fc12e8f0 legacy=0x7f58fc130cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f58faffd640 1 -- 172.21.15.177:0/2053083300 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0940f0 con 0x7f58fc068260 2024-09-13T18:35:14.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 -- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 msgr2=0x7f1330126cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 --2- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330126cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 -- 172.21.15.177:0/2939803537 >> v1:172.21.15.177:6789/0 conn(0x7f1330064080 legacy=0x7f1330069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f1334eb8640 1 -- 172.21.15.177:0/2939803537 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330098860 con 0x7f1330064450 2024-09-13T18:35:14.843 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f2fb3fff640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb412fad0 0x7f2fb4131eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.843 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.825+0000 7f20822b9640 1 -- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c12b0e0 msgr2=0x7f207c12d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.843 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f20822b9640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c12b0e0 0x7f207c12d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f20822b9640 1 -- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 msgr2=0x7f207c091d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f20822b9640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 0x7f207c091d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f20822b9640 1 -- 172.21.15.177:0/3688157889 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c090e30 con 0x7f207c068120 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f2082aba640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c12b0e0 0x7f207c12d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f2081ab8640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 0x7f207c091d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f2fb37fe640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb4068260 0x7f2fb4068630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098b0 tx=0x7f2fa40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348e602d0c5f8d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f58faffd640 1 --2- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068260 0x7f58fc068630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009850 tx=0x7f58ec0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be7772342e1621a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f2fb27fc640 1 -- 172.21.15.177:0/1348833051 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa402ac30 con 0x7f2fb4068260 2024-09-13T18:35:14.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f2fb27fc640 1 -- 172.21.15.177:0/1348833051 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa402ad90 con 0x7f2fb4068260 2024-09-13T18:35:14.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f2fb27fc640 1 -- 172.21.15.177:0/1348833051 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4032670 con 0x7f2fb4068260 2024-09-13T18:35:14.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f58fa7fc640 1 -- 172.21.15.177:0/2053083300 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec02ac70 con 0x7f58fc068260 2024-09-13T18:35:14.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f58fa7fc640 1 -- 172.21.15.177:0/2053083300 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec02add0 con 0x7f58fc068260 2024-09-13T18:35:14.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f58fa7fc640 1 -- 172.21.15.177:0/2053083300 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec032670 con 0x7f58fc068260 2024-09-13T18:35:14.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.826+0000 7f1334eb8640 1 --2- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f133006a380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1320009940 tx=0x7f13200091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320a9c0164e69390 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f20822b9640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c068120 0x7f207c091450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2078009910 tx=0x7f20780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a370547db9b7599d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f20812b7640 1 -- 172.21.15.177:0/3688157889 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207802ac90 con 0x7f207c068120 2024-09-13T18:35:14.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f20812b7640 1 -- 172.21.15.177:0/3688157889 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207802adf0 con 0x7f207c068120 2024-09-13T18:35:14.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f20812b7640 1 -- 172.21.15.177:0/3688157889 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078032670 con 0x7f207c068120 2024-09-13T18:35:14.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f1327fff640 1 -- 172.21.15.177:0/2939803537 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132002ac70 con 0x7f1330064450 2024-09-13T18:35:14.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f1327fff640 1 -- 172.21.15.177:0/2939803537 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132002add0 con 0x7f1330064450 2024-09-13T18:35:14.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f1327fff640 1 -- 172.21.15.177:0/2939803537 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320032670 con 0x7f1330064450 2024-09-13T18:35:14.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 -- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb4068260 msgr2=0x7f2fb4068630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb4068260 0x7f2fb4068630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098b0 tx=0x7f2fa40090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 -- 172.21.15.177:0/1348833051 shutdown_connections 2024-09-13T18:35:14.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb412fad0 0x7f2fb4131eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1348833051 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb4068260 0x7f2fb4068630 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 -- 172.21.15.177:0/1348833051 >> 172.21.15.177:0/1348833051 conn(0x7f2fb4099d30 msgr2=0x7f2fb409a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 -- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068260 msgr2=0x7f58fc068630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 --2- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068260 0x7f58fc068630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009850 tx=0x7f58ec0090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 -- 172.21.15.177:0/1348833051 shutdown_connections 2024-09-13T18:35:14.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 -- 172.21.15.177:0/2053083300 shutdown_connections 2024-09-13T18:35:14.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 --2- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068b70 0x7f58fc12e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 --2- 172.21.15.177:0/2053083300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068260 0x7f58fc068630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 -- 172.21.15.177:0/2053083300 >> 172.21.15.177:0/2053083300 conn(0x7f58fc099d30 msgr2=0x7f58fc09a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f2fba24f640 1 -- 172.21.15.177:0/1348833051 wait complete. 2024-09-13T18:35:14.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.827+0000 7f590148b640 1 -- 172.21.15.177:0/2053083300 shutdown_connections 2024-09-13T18:35:14.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f590148b640 1 -- 172.21.15.177:0/2053083300 wait complete. 2024-09-13T18:35:14.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 -- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 msgr2=0x7f133006a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 --2- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f133006a380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1320009940 tx=0x7f13200091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c068120 msgr2=0x7f207c091450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 -- 172.21.15.177:0/2939803537 shutdown_connections 2024-09-13T18:35:14.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c068120 0x7f207c091450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2078009910 tx=0x7f20780091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 --2- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330126cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 --2- 172.21.15.177:0/2939803537 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f133006a380 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 -- 172.21.15.177:0/2939803537 >> 172.21.15.177:0/2939803537 conn(0x7f1330096210 msgr2=0x7f1330096610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 -- 172.21.15.177:0/2939803537 shutdown_connections 2024-09-13T18:35:14.852 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f133791d640 1 -- 172.21.15.177:0/2939803537 wait complete. 2024-09-13T18:35:14.853 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 shutdown_connections 2024-09-13T18:35:14.853 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c12b0e0 0x7f207c12d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.853 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c091990 0x7f207c091d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.853 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 --2- 172.21.15.177:0/3688157889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c068120 0x7f207c091450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.853 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 >> 172.21.15.177:0/3688157889 conn(0x7f207c097e70 msgr2=0x7f207c098270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.853 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 shutdown_connections 2024-09-13T18:35:14.854 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.828+0000 7f208451d640 1 -- 172.21.15.177:0/3688157889 wait complete. 2024-09-13T18:35:14.854 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 Processor -- start 2024-09-13T18:35:14.854 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 -- start start 2024-09-13T18:35:14.854 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f2fba24f640 1 Processor -- start 2024-09-13T18:35:14.854 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f2fba24f640 1 -- start start 2024-09-13T18:35:14.855 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f208451d640 1 Processor -- start 2024-09-13T18:35:14.855 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068260 0x7f58fc08d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.855 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068b70 0x7f58fc08dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.855 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f133791d640 1 Processor -- start 2024-09-13T18:35:14.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f58fc12e8f0 0x7f58fc08e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f208451d640 1 -- start start 2024-09-13T18:35:14.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc090c60 con 0x7f58fc068b70 2024-09-13T18:35:14.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc08fb50 con 0x7f58fc068260 2024-09-13T18:35:14.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f590148b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc09ca10 con 0x7f58fc12e8f0 2024-09-13T18:35:14.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f133791d640 1 -- start start 2024-09-13T18:35:14.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068b70 0x7f58fc08dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f58fb7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f58fc12e8f0 0x7f58fc08e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f58faffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068260 0x7f58fc08d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068b70 0x7f58fc08dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41582/0 (socket says 172.21.15.177:41582) 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.829+0000 7f58f3fff640 1 -- 172.21.15.177:0/3383485837 learned_addr learned my addr 172.21.15.177:0/3383485837 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58fb7fe640 1 -- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068260 msgr2=0x7f58fc08d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58fb7fe640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068260 0x7f58fc08d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58fb7fe640 1 -- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068b70 msgr2=0x7f58fc08dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58fb7fe640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068b70 0x7f58fc08dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58fb7fe640 1 -- 172.21.15.177:0/3383485837 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec009390 con 0x7f58fc12e8f0 2024-09-13T18:35:14.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fba24f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 0x7f2fb408dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58faffd640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068260 0x7f58fc08d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:14.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fba24f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fb4068b70 0x7f2fb408e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fba24f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb412fad0 0x7f2fb408e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fba24f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4090640 con 0x7f2fb4068b70 2024-09-13T18:35:14.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fba24f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4122440 con 0x7f2fb4068260 2024-09-13T18:35:14.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58fb7fe640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f58fc12e8f0 0x7f58fc08e260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f58fc097870 tx=0x7f58f4002660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 0x7f2fb408dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 0x7f2fb408dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41614/0 (socket says 172.21.15.177:41614) 2024-09-13T18:35:14.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1142700671 learned_addr learned my addr 172.21.15.177:0/1142700671 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fb3fff640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb412fad0 0x7f2fb408e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fba24f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4122750 con 0x7f2fb412fad0 2024-09-13T18:35:14.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f208451d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 0x7f207c088280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2fb2ffd640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fb4068b70 0x7f2fb408e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f208451d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 0x7f207c0887c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f133791d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330064080 0x7f1330126c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f133791d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f13301271d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f208451d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c12b0e0 0x7f207c135e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f20822b9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 0x7f207c088280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f20822b9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 0x7f207c088280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51270/0 (socket says 172.21.15.177:51270) 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f2081ab8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 0x7f207c0887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2081ab8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 0x7f207c0887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41622/0 (socket says 172.21.15.177:41622) 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400a5e0 con 0x7f58fc12e8f0 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400a760 con 0x7f58fc12e8f0 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400ab20 con 0x7f58fc12e8f0 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58fc12e0e0 con 0x7f58fc12e8f0 2024-09-13T18:35:14.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58fc08e7a0 con 0x7f58fc12e8f0 2024-09-13T18:35:14.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f20822b9640 1 -- 172.21.15.177:0/3677222260 learned_addr learned my addr 172.21.15.177:0/3677222260 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb412fad0 msgr2=0x7f2fb408e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb37fe640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb412fad0 0x7f2fb408e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fb4068b70 msgr2=0x7f2fb408e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb37fe640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fb4068b70 0x7f2fb408e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f208451d640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c08aa90 con 0x7f207c091990 2024-09-13T18:35:14.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f208451d640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c087fc0 con 0x7f207c12b0e0 2024-09-13T18:35:14.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f208451d640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f207c06eca0 con 0x7f207c068120 2024-09-13T18:35:14.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb37fe640 1 -- 172.21.15.177:0/1142700671 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009a20 con 0x7f2fb4068260 2024-09-13T18:35:14.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb2ffd640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fb4068b70 0x7f2fb408e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:14.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c12b0e0 0x7f207c135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fb37fe640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 0x7f2fb408dd60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2fa402a5a0 tx=0x7f2fa4032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 -- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 msgr2=0x7f207c088280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 0x7f207c088280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 -- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 msgr2=0x7f207c0887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 0x7f207c0887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c009390 con 0x7f207c12b0e0 2024-09-13T18:35:14.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031700 con 0x7f2fb4068260 2024-09-13T18:35:14.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4031860 con 0x7f2fb4068260 2024-09-13T18:35:14.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031be0 con 0x7f2fb4068260 2024-09-13T18:35:14.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa4009390 con 0x7f2fb4068260 2024-09-13T18:35:14.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb412f6a0 con 0x7f2fb4068260 2024-09-13T18:35:14.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f2082aba640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c12b0e0 0x7f207c135e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f207400e650 tx=0x7f207400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f20822b9640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 0x7f207c088280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:35:14.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f2081ab8640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 0x7f207c0887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:35:14.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074012640 con 0x7f207c12b0e0 2024-09-13T18:35:14.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20740127c0 con 0x7f207c12b0e0 2024-09-13T18:35:14.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f133791d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330127710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:14.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f133791d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1330129b30 con 0x7f1330064080 2024-09-13T18:35:14.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f133791d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133012f110 con 0x7f1330126870 2024-09-13T18:35:14.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f133791d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f133012f650 con 0x7f1330064450 2024-09-13T18:35:14.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f1335eba640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330127710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f1335eba640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330127710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41634/0 (socket says 172.21.15.177:41634) 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.831+0000 7f1335eba640 1 -- 172.21.15.177:0/3597675340 learned_addr learned my addr 172.21.15.177:0/3597675340 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1335eba640 1 -- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 msgr2=0x7f13301271d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1334eb8640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f13301271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.830+0000 7f13356b9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330064080 0x7f1330126c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1335eba640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f13301271d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1335eba640 1 -- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330064080 msgr2=0x7f1330126c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1335eba640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330064080 0x7f1330126c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1335eba640 1 -- 172.21.15.177:0/3597675340 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009450 con 0x7f1330126870 2024-09-13T18:35:14.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f208451d640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2074009a20 con 0x7f207c12b0e0 2024-09-13T18:35:14.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f13356b9640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330064080 0x7f1330126c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:35:14.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f208451d640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2078009450 con 0x7f207c12b0e0 2024-09-13T18:35:14.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.832+0000 7f1335eba640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330127710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1328013a90 tx=0x7f13280143c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:14.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb405ed70 con 0x7f2fb4068260 2024-09-13T18:35:14.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13280112e0 con 0x7f1330126870 2024-09-13T18:35:14.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1328011440 con 0x7f1330126870 2024-09-13T18:35:14.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1330126490 con 0x7f1330126870 2024-09-13T18:35:14.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13280117e0 con 0x7f1330126870 2024-09-13T18:35:14.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f2fa4039040 con 0x7f2fb4068260 2024-09-13T18:35:14.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133012faa0 con 0x7f1330126870 2024-09-13T18:35:14.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f2fa4083240 con 0x7f2fb4068260 2024-09-13T18:35:14.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.833+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074012b80 con 0x7f207c12b0e0 2024-09-13T18:35:14.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.834+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58fc05ed70 con 0x7f58fc12e8f0 2024-09-13T18:35:14.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.834+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f58f4012420 con 0x7f58fc12e8f0 2024-09-13T18:35:14.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.834+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f207401a420 con 0x7f207c12b0e0 2024-09-13T18:35:14.877 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.834+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f58f4063400 con 0x7f58fc12e8f0 2024-09-13T18:35:14.877 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.835+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c06f440 con 0x7f207c12b0e0 2024-09-13T18:35:14.877 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.835+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133012fd80 con 0x7f1330126870 2024-09-13T18:35:14.877 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.835+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f2074016070 con 0x7f207c12b0e0 2024-09-13T18:35:14.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.837+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f2fa405e930 con 0x7f2fb4068260 2024-09-13T18:35:14.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.838+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f58f403e5a0 con 0x7f58fc12e8f0 2024-09-13T18:35:14.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.839+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f1328021400 con 0x7f1330126870 2024-09-13T18:35:14.878 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.841+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f132804b890 con 0x7f1330126870 2024-09-13T18:35:14.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.841+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f2074041490 con 0x7f207c12b0e0 2024-09-13T18:35:14.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.847+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f1328047ce0 con 0x7f1330126870 2024-09-13T18:35:14.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.974+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2fb4098520 con 0x7f2fb4068260 2024-09-13T18:35:14.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.975+0000 7f2f9b7fe640 1 -- 172.21.15.177:0/1142700671 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fa403f070 con 0x7f2fb4068260 2024-09-13T18:35:14.978 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:14.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.976+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f58fc09c190 con 0x7f58fc12e8f0 2024-09-13T18:35:14.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.977+0000 7f58f2ffd640 1 -- 172.21.15.177:0/3383485837 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58f4016070 con 0x7f58fc12e8f0 2024-09-13T18:35:14.980 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:14.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 >> v1:172.21.15.177:6816/32265 conn(0x7f2fa004cbd0 legacy=0x7f2fa004f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 msgr2=0x7f2fb408dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 0x7f2fb408dd60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2fa402a5a0 tx=0x7f2fa4032000 comp rx=0 tx=0).stop 2024-09-13T18:35:14.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 shutdown_connections 2024-09-13T18:35:14.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2fb412fad0 0x7f2fb408e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fb4068b70 0x7f2fb408e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 --2- 172.21.15.177:0/1142700671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2fb4068260 0x7f2fb408dd60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 >> 172.21.15.177:0/1142700671 conn(0x7f2fb4099d30 msgr2=0x7f2fb4096730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 shutdown_connections 2024-09-13T18:35:14.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.978+0000 7f2fba24f640 1 -- 172.21.15.177:0/1142700671 wait complete. 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 >> v1:172.21.15.177:6816/32265 conn(0x7f58dc04cc40 legacy=0x7f58dc04f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f58fc12e8f0 msgr2=0x7f58fc08e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f58fc12e8f0 0x7f58fc08e260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f58fc097870 tx=0x7f58f4002660 comp rx=0 tx=0).stop 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 shutdown_connections 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f58fc12e8f0 0x7f58fc08e260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58fc068b70 0x7f58fc08dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 --2- 172.21.15.177:0/3383485837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58fc068260 0x7f58fc08d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 >> 172.21.15.177:0/3383485837 conn(0x7f58fc099d30 msgr2=0x7f58fc092b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 shutdown_connections 2024-09-13T18:35:14.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.979+0000 7f590148b640 1 -- 172.21.15.177:0/3383485837 wait complete. 2024-09-13T18:35:14.987 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.984+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f207c05ed70 con 0x7f207c12b0e0 2024-09-13T18:35:14.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.985+0000 7f20727fc640 1 -- 172.21.15.177:0/3677222260 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2074044ca0 con 0x7f207c12b0e0 2024-09-13T18:35:14.988 INFO:teuthology.orchestra.run.smithi177.stdout:38654705667 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 >> v1:172.21.15.177:6816/32265 conn(0x7f206004ccc0 legacy=0x7f206004f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c12b0e0 msgr2=0x7f207c135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c12b0e0 0x7f207c135e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f207400e650 tx=0x7f207400b040 comp rx=0 tx=0).stop 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 shutdown_connections 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f207c12b0e0 0x7f207c135e20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f207c091990 0x7f207c0887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 --2- 172.21.15.177:0/3677222260 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f207c068120 0x7f207c088280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 >> 172.21.15.177:0/3677222260 conn(0x7f207c097e70 msgr2=0x7f207c1282e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 shutdown_connections 2024-09-13T18:35:14.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.987+0000 7f205ffff640 1 -- 172.21.15.177:0/3677222260 wait complete. 2024-09-13T18:35:14.994 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.992+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1330129120 con 0x7f1330126870 2024-09-13T18:35:14.995 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.992+0000 7f13257fa640 1 -- 172.21.15.177:0/3597675340 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f132801c070 con 0x7f1330126870 2024-09-13T18:35:14.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2024-09-13T18:35:14.996 INFO:teuthology.orchestra.run.smithi177.stdout:38654705667 2024-09-13T18:35:14.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-09-13T18:35:14.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 >> v1:172.21.15.177:6816/32265 conn(0x7f131404cb70 legacy=0x7f131404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 msgr2=0x7f1330127710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330127710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1328013a90 tx=0x7f13280143c0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 shutdown_connections 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1330126870 0x7f1330127710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1330064450 0x7f13301271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 --2- 172.21.15.177:0/3597675340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330064080 0x7f1330126c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 >> 172.21.15.177:0/3597675340 conn(0x7f1330096210 msgr2=0x7f133006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 shutdown_connections 2024-09-13T18:35:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:14.994+0000 7f133791d640 1 -- 172.21.15.177:0/3597675340 wait complete. 2024-09-13T18:35:15.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.1 2024-09-13T18:35:15.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-09-13T18:35:15.997 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:35:15.997 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:35:16.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:35:16.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:35:16.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.103+0000 7f3ad479d640 1 Processor -- start 2024-09-13T18:35:16.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.103+0000 7fc0008b8640 1 Processor -- start 2024-09-13T18:35:16.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.103+0000 7fc0008b8640 1 -- start start 2024-09-13T18:35:16.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.103+0000 7f3ad479d640 1 -- start start 2024-09-13T18:35:16.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fc0008b8640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc062d90 0x7fbffc063160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fc0008b8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbffc0636a0 0x7fbffc12a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fc0008b8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc12a630 0x7fbffc12aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad479d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc098630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fc0008b8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc093660 con 0x7fbffc0636a0 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fc0008b8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc068b10 con 0x7fbffc12a630 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fc0008b8640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc068330 con 0x7fbffc062d90 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad479d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc098b70 0x7f3acc098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad479d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 0x7f3acc130cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad479d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc06a960 con 0x7f3acc12e8d0 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad479d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc06eea0 con 0x7f3acc093cc0 2024-09-13T18:35:16.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad479d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0970d0 con 0x7f3acc098b70 2024-09-13T18:35:16.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc12a630 0x7fbffc12aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fbff9d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbffc0636a0 0x7fbffc12a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc062d90 0x7fbffc063160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fbff9d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbffc0636a0 0x7fbffc12a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41626/0 (socket says 172.21.15.177:41626) 2024-09-13T18:35:16.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7fbff9d9b640 1 -- 172.21.15.177:0/3650117101 learned_addr learned my addr 172.21.15.177:0/3650117101 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad2539640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc098630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad2d3a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 0x7f3acc130cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad2d3a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 0x7f3acc130cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41628/0 (socket says 172.21.15.177:41628) 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad2d3a640 1 -- 172.21.15.177:0/2235046036 learned_addr learned my addr 172.21.15.177:0/2235046036 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.104+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc098b70 0x7f3acc098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7f3ad1d38640 1 -- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 msgr2=0x7f3acc098630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc098630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7f3ad1d38640 1 -- 172.21.15.177:0/2235046036 >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 msgr2=0x7f3acc130cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/2235046036 >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 0x7f3acc130cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7f3ad1d38640 1 -- 172.21.15.177:0/2235046036 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc099480 con 0x7f3acc098b70 2024-09-13T18:35:16.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7fbffad9d640 1 -- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc062d90 msgr2=0x7fbffc063160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7fbffad9d640 1 --2- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc062d90 0x7fbffc063160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7fbffad9d640 1 -- 172.21.15.177:0/3650117101 >> v2:172.21.15.177:3300/0 conn(0x7fbffc0636a0 msgr2=0x7fbffc12a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7fbffad9d640 1 --2- 172.21.15.177:0/3650117101 >> v2:172.21.15.177:3300/0 conn(0x7fbffc0636a0 0x7fbffc12a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7fbffad9d640 1 -- 172.21.15.177:0/3650117101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc06f300 con 0x7fbffc12a630 2024-09-13T18:35:16.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.105+0000 7f3ad2d3a640 1 --2- 172.21.15.177:0/2235046036 >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 0x7f3acc130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:16.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7fbffad9d640 1 --2- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc12a630 0x7fbffc12aad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009e60 tx=0x7fbff4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2bb977228e9454 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc098b70 0x7f3acc098f40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009880 tx=0x7f3abc0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a97ea48ecce77a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7fbff959a640 1 -- 172.21.15.177:0/3650117101 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4007550 con 0x7fbffc12a630 2024-09-13T18:35:16.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7fbff959a640 1 -- 172.21.15.177:0/3650117101 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff402d050 con 0x7fbffc12a630 2024-09-13T18:35:16.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7fbff959a640 1 -- 172.21.15.177:0/3650117101 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4007920 con 0x7fbffc12a630 2024-09-13T18:35:16.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7f3ad1537640 1 -- 172.21.15.177:0/2235046036 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc02abc0 con 0x7f3acc098b70 2024-09-13T18:35:16.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7f3ad1537640 1 -- 172.21.15.177:0/2235046036 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc02ad20 con 0x7f3acc098b70 2024-09-13T18:35:16.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.106+0000 7f3ad1537640 1 -- 172.21.15.177:0/2235046036 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc032670 con 0x7f3acc098b70 2024-09-13T18:35:16.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 -- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc12a630 msgr2=0x7fbffc12aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 --2- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc12a630 0x7fbffc12aad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009e60 tx=0x7fbff4009780 comp rx=0 tx=0).stop 2024-09-13T18:35:16.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 -- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc098b70 msgr2=0x7f3acc098f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 --2- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc098b70 0x7f3acc098f40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009880 tx=0x7f3abc0090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 -- 172.21.15.177:0/3650117101 shutdown_connections 2024-09-13T18:35:16.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 --2- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc12a630 0x7fbffc12aad0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 --2- 172.21.15.177:0/3650117101 >> v2:172.21.15.177:3300/0 conn(0x7fbffc0636a0 0x7fbffc12a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 --2- 172.21.15.177:0/3650117101 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc062d90 0x7fbffc063160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 -- 172.21.15.177:0/3650117101 >> 172.21.15.177:0/3650117101 conn(0x7fbffc097e70 msgr2=0x7fbffc098270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 -- 172.21.15.177:0/2235046036 shutdown_connections 2024-09-13T18:35:16.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 --2- 172.21.15.177:0/2235046036 >> v2:172.21.15.177:3300/0 conn(0x7f3acc12e8d0 0x7f3acc130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 --2- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc098b70 0x7f3acc098f40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 --2- 172.21.15.177:0/2235046036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc098630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 -- 172.21.15.177:0/2235046036 >> 172.21.15.177:0/2235046036 conn(0x7f3acc096210 msgr2=0x7f3acc096610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 -- 172.21.15.177:0/3650117101 shutdown_connections 2024-09-13T18:35:16.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7fc0008b8640 1 -- 172.21.15.177:0/3650117101 wait complete. 2024-09-13T18:35:16.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 -- 172.21.15.177:0/2235046036 shutdown_connections 2024-09-13T18:35:16.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.107+0000 7f3ad479d640 1 -- 172.21.15.177:0/2235046036 wait complete. 2024-09-13T18:35:16.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 Processor -- start 2024-09-13T18:35:16.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 -- start start 2024-09-13T18:35:16.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 Processor -- start 2024-09-13T18:35:16.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 -- start start 2024-09-13T18:35:16.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc062d90 0x7fbffc1391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 0x7fbffc139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 0x7fbffc139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc12a050 con 0x7fbffc12a630 2024-09-13T18:35:16.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc090830 con 0x7fbffc062d90 2024-09-13T18:35:16.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fc0008b8640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0900b0 con 0x7fbffc0636a0 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 0x7f3acc1394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc12e8d0 0x7f3acc11e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fbff9d9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 0x7fbffc139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 0x7fbffc139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fbff9d9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 0x7fbffc139720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51306/0 (socket says 172.21.15.177:51306) 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fbff9d9b640 1 -- 172.21.15.177:0/4222595451 learned_addr learned my addr 172.21.15.177:0/4222595451 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fbffad9d640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 0x7fbffc139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41636/0 (socket says 172.21.15.177:41636) 2024-09-13T18:35:16.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7fbffa59c640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc062d90 0x7fbffc1391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc1204f0 con 0x7f3acc098b70 2024-09-13T18:35:16.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc1210a0 con 0x7f3acc093cc0 2024-09-13T18:35:16.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.108+0000 7f3ad479d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc092790 con 0x7f3acc12e8d0 2024-09-13T18:35:16.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 0x7f3acc1394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 0x7f3acc1394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41646/0 (socket says 172.21.15.177:41646) 2024-09-13T18:35:16.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 -- 172.21.15.177:0/3428368772 learned_addr learned my addr 172.21.15.177:0/3428368772 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad2d3a640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc12e8d0 0x7f3acc11e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad2539640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbff9d9b640 1 -- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc062d90 msgr2=0x7fbffc1391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbff9d9b640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc062d90 0x7fbffc1391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbff9d9b640 1 -- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 msgr2=0x7fbffc139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbff9d9b640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 0x7fbffc139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbff9d9b640 1 -- 172.21.15.177:0/4222595451 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009a20 con 0x7fbffc0636a0 2024-09-13T18:35:16.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 -- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc12e8d0 msgr2=0x7f3acc11e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc12e8d0 0x7f3acc11e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 -- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 msgr2=0x7f3acc138fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc138fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 -- 172.21.15.177:0/3428368772 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4009ae0 con 0x7f3acc098b70 2024-09-13T18:35:16.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad1d38640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 0x7f3acc1394e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009880 tx=0x7f3abc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbffad9d640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 0x7fbffc139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:16.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc032e00 con 0x7f3acc098b70 2024-09-13T18:35:16.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7fbff9d9b640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 0x7fbffc139720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbff000ea20 tx=0x7fbff00091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.109+0000 7f3ad479d640 1 -- 172.21.15.177:0/3428368772 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc009390 con 0x7f3acc098b70 2024-09-13T18:35:16.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc02ae90 con 0x7f3acc098b70 2024-09-13T18:35:16.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7f3ad479d640 1 -- 172.21.15.177:0/3428368772 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc139a20 con 0x7f3acc098b70 2024-09-13T18:35:16.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00095a0 con 0x7fbffc0636a0 2024-09-13T18:35:16.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff000c940 con 0x7fbffc0636a0 2024-09-13T18:35:16.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc031c70 con 0x7f3acc098b70 2024-09-13T18:35:16.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc1208f0 con 0x7fbffc0636a0 2024-09-13T18:35:16.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.110+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc11f260 con 0x7fbffc0636a0 2024-09-13T18:35:16.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.111+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000cce0 con 0x7fbffc0636a0 2024-09-13T18:35:16.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.111+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f3abc039040 con 0x7f3acc098b70 2024-09-13T18:35:16.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.112+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa8005210 con 0x7f3acc098b70 2024-09-13T18:35:16.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.112+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc1210f0 con 0x7fbffc0636a0 2024-09-13T18:35:16.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.112+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f3abc083030 con 0x7f3acc098b70 2024-09-13T18:35:16.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.112+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fbff0013680 con 0x7fbffc0636a0 2024-09-13T18:35:16.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.113+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fbff0063420 con 0x7fbffc0636a0 2024-09-13T18:35:16.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.116+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fbff003e330 con 0x7fbffc0636a0 2024-09-13T18:35:16.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.116+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f3abc0ac270 con 0x7f3acc098b70 2024-09-13T18:35:16.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.144+0000 7f80b479d640 1 Processor -- start 2024-09-13T18:35:16.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.144+0000 7ff63fd5d640 1 Processor -- start 2024-09-13T18:35:16.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.144+0000 7f80b479d640 1 -- start start 2024-09-13T18:35:16.149 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.144+0000 7ff63fd5d640 1 -- start start 2024-09-13T18:35:16.149 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b479d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac098e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.150 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b479d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac128670 0x7f80ac12fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.150 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63fd5d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff638068260 0x7ff638068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.150 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63fd5d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff638068b70 0x7ff63812e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.150 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63fd5d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 0x7ff638130cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.151 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b479d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac098630 con 0x7f80ac127b70 2024-09-13T18:35:16.151 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 0x7ff638130cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.151 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63fd5d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff638069370 con 0x7ff638068260 2024-09-13T18:35:16.152 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b479d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac06a200 con 0x7f80ac128670 2024-09-13T18:35:16.152 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63fd5d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380940f0 con 0x7ff63812e8f0 2024-09-13T18:35:16.152 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63fd5d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff638069da0 con 0x7ff638068b70 2024-09-13T18:35:16.152 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 0x7ff638130cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41658/0 (socket says 172.21.15.177:41658) 2024-09-13T18:35:16.152 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 -- 172.21.15.177:0/2750442633 learned_addr learned my addr 172.21.15.177:0/2750442633 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.153 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b479d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac06a440 con 0x7f80ac098a60 2024-09-13T18:35:16.153 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b2d3a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac128670 0x7f80ac12fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.153 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b1d38640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f80ac127b70 0x7f80ac127f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47196/0 (socket says 172.21.15.177:47196) 2024-09-13T18:35:16.153 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b1d38640 1 -- 172.21.15.177:0/2777652810 learned_addr learned my addr 172.21.15.177:0/2777652810 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b2539640 1 --2- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac098e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63d2f8640 1 --2- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff638068b70 0x7ff63812e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 -- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff638068b70 msgr2=0x7ff63812e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 --2- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff638068b70 0x7ff63812e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 -- 172.21.15.177:0/2750442633 >> v2:172.21.15.177:3300/0 conn(0x7ff638068260 msgr2=0x7ff638068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 --2- 172.21.15.177:0/2750442633 >> v2:172.21.15.177:3300/0 conn(0x7ff638068260 0x7ff638068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.154 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7ff63e2fa640 1 -- 172.21.15.177:0/2750442633 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380994e0 con 0x7ff63812e8f0 2024-09-13T18:35:16.155 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b2d3a640 1 -- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 msgr2=0x7f80ac098e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:16.155 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b2d3a640 1 --2- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac098e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.155 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b2d3a640 1 -- 172.21.15.177:0/2777652810 >> v1:172.21.15.177:6789/0 conn(0x7f80ac127b70 legacy=0x7f80ac127f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.155 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.145+0000 7f80b2d3a640 1 -- 172.21.15.177:0/2777652810 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac06cf90 con 0x7f80ac128670 2024-09-13T18:35:16.156 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.146+0000 7ff63e2fa640 1 --2- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 0x7ff638130cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff630009f50 tx=0x7ff630009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f613bce03b5f48f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.156 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.146+0000 7f80b2d3a640 1 --2- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac128670 0x7f80ac12fec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009f10 tx=0x7f80a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5030dacd2e177f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.156 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.146+0000 7ff63caf7640 1 -- 172.21.15.177:0/2750442633 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630031400 con 0x7ff63812e8f0 2024-09-13T18:35:16.156 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.146+0000 7ff63caf7640 1 -- 172.21.15.177:0/2750442633 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63002d050 con 0x7ff63812e8f0 2024-09-13T18:35:16.157 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.146+0000 7ff63caf7640 1 -- 172.21.15.177:0/2750442633 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6300317d0 con 0x7ff63812e8f0 2024-09-13T18:35:16.157 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.146+0000 7f80b1537640 1 -- 172.21.15.177:0/2777652810 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4032640 con 0x7f80ac128670 2024-09-13T18:35:16.157 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7f80b1537640 1 -- 172.21.15.177:0/2777652810 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a40327a0 con 0x7f80ac128670 2024-09-13T18:35:16.157 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7f80b1537640 1 -- 172.21.15.177:0/2777652810 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4032b80 con 0x7f80ac128670 2024-09-13T18:35:16.158 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 msgr2=0x7ff638130cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.158 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 0x7ff638130cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff630009f50 tx=0x7ff630009840 comp rx=0 tx=0).stop 2024-09-13T18:35:16.158 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2750442633 shutdown_connections 2024-09-13T18:35:16.158 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63812e8f0 0x7ff638130cd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2750442633 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff638068b70 0x7ff63812e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2750442633 >> v2:172.21.15.177:3300/0 conn(0x7ff638068260 0x7ff638068630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2750442633 >> 172.21.15.177:0/2750442633 conn(0x7ff638099d30 msgr2=0x7ff63809a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2750442633 shutdown_connections 2024-09-13T18:35:16.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2750442633 wait complete. 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7f80b479d640 1 -- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac128670 msgr2=0x7f80ac12fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.147+0000 7f80b479d640 1 --2- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac128670 0x7f80ac12fec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009f10 tx=0x7f80a4009780 comp rx=0 tx=0).stop 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7f80b479d640 1 -- 172.21.15.177:0/2777652810 shutdown_connections 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7f80b479d640 1 --2- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac128670 0x7f80ac12fec0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7f80b479d640 1 --2- 172.21.15.177:0/2777652810 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac098e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7f80b479d640 1 -- 172.21.15.177:0/2777652810 >> 172.21.15.177:0/2777652810 conn(0x7f80ac096210 msgr2=0x7f80ac096610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7f80b479d640 1 -- 172.21.15.177:0/2777652810 shutdown_connections 2024-09-13T18:35:16.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7f80b479d640 1 -- 172.21.15.177:0/2777652810 wait complete. 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7ff63fd5d640 1 Processor -- start 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.148+0000 7ff63fd5d640 1 -- start start 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63fd5d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 0x7ff638139050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63fd5d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff638068b70 0x7ff638139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63fd5d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff63812e8f0 0x7ff638139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63fd5d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6381261d0 con 0x7ff638068b70 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63fd5d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6381248e0 con 0x7ff638068260 2024-09-13T18:35:16.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63fd5d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6381264b0 con 0x7ff63812e8f0 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 0x7ff638139050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 0x7ff638139050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41674/0 (socket says 172.21.15.177:41674) 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 -- 172.21.15.177:0/2858015793 learned_addr learned my addr 172.21.15.177:0/2858015793 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63d2f8640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff638068b70 0x7ff638139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63e2fa640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff63812e8f0 0x7ff638139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 -- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff63812e8f0 msgr2=0x7ff638139ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff63812e8f0 0x7ff638139ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 -- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff638068b70 msgr2=0x7ff638139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff638068b70 0x7ff638139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 -- 172.21.15.177:0/2858015793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630009ae0 con 0x7ff638068260 2024-09-13T18:35:16.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63d2f8640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff638068b70 0x7ff638139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:16.164 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7f80b479d640 1 Processor -- start 2024-09-13T18:35:16.164 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff63daf9640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 0x7ff638139050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff63400b810 tx=0x7ff6340091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.164 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7f80b479d640 1 -- start start 2024-09-13T18:35:16.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.149+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63400be60 con 0x7ff638068260 2024-09-13T18:35:16.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63400c730 con 0x7ff638068260 2024-09-13T18:35:16.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63400caf0 con 0x7ff638068260 2024-09-13T18:35:16.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6381269b0 con 0x7ff638068260 2024-09-13T18:35:16.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff638123840 con 0x7ff638068260 2024-09-13T18:35:16.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7f80b479d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7f80b479d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 0x7f80ac135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.150+0000 7f80b479d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac128670 0x7f80ac087fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b2539640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b1d38640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 0x7f80ac135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b2539640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac134ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51350/0 (socket says 172.21.15.177:51350) 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b2d3a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac128670 0x7f80ac087fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b1d38640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 0x7f80ac135400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41676/0 (socket says 172.21.15.177:41676) 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6381250a0 con 0x7ff638068260 2024-09-13T18:35:16.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7ff634012070 con 0x7ff638068260 2024-09-13T18:35:16.171 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac08b640 con 0x7f80ac128670 2024-09-13T18:35:16.171 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b2539640 1 -- 172.21.15.177:0/1959272774 learned_addr learned my addr 172.21.15.177:0/1959272774 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:16.172 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac08b190 con 0x7f80ac127b70 2024-09-13T18:35:16.172 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac08a260 con 0x7f80ac098a60 2024-09-13T18:35:16.172 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.151+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7ff634062700 con 0x7ff638068260 2024-09-13T18:35:16.173 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2539640 1 -- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 msgr2=0x7f80ac135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2539640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 0x7f80ac135400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2539640 1 -- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac128670 msgr2=0x7f80ac087fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2539640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac128670 0x7f80ac087fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2539640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4009a20 con 0x7f80ac098a60 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2d3a640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac128670 0x7f80ac087fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b1d38640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 0x7f80ac135400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.152+0000 7f80b2539640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac134ec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d890 tx=0x7f80a8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.153+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800de20 con 0x7f80ac098a60 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.153+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac08aa90 con 0x7f80ac098a60 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.153+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a800b8d0 con 0x7f80ac098a60 2024-09-13T18:35:16.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.153+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac135ae0 con 0x7f80ac098a60 2024-09-13T18:35:16.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.154+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800de20 con 0x7f80ac098a60 2024-09-13T18:35:16.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.154+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac05ed70 con 0x7f80ac098a60 2024-09-13T18:35:16.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.155+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7ff63403d840 con 0x7ff638068260 2024-09-13T18:35:16.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.155+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f80a800ba40 con 0x7f80ac098a60 2024-09-13T18:35:16.176 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.156+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f80a80427e0 con 0x7f80ac098a60 2024-09-13T18:35:16.176 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.161+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f80a803f560 con 0x7f80ac098a60 2024-09-13T18:35:16.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.260+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbffc05ed70 con 0x7fbffc0636a0 2024-09-13T18:35:16.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.260+0000 7fbfea7fc640 1 -- 172.21.15.177:0/4222595451 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbff000ce40 con 0x7fbffc0636a0 2024-09-13T18:35:16.263 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:16.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.261+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3aa80054c0 con 0x7f3acc098b70 2024-09-13T18:35:16.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.261+0000 7f3ac27fc640 1 -- 172.21.15.177:0/3428368772 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3abc042020 con 0x7f3acc098b70 2024-09-13T18:35:16.264 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 >> v1:172.21.15.177:6816/32265 conn(0x7fbfdc04ccc0 legacy=0x7fbfdc04f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 msgr2=0x7fbffc139720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 0x7fbffc139720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbff000ea20 tx=0x7fbff00091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 shutdown_connections 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc12a630 0x7fbffc139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fbffc0636a0 0x7fbffc139720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 --2- 172.21.15.177:0/4222595451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbffc062d90 0x7fbffc1391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 >> 172.21.15.177:0/4222595451 conn(0x7fbffc097e70 msgr2=0x7fbffc0923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 shutdown_connections 2024-09-13T18:35:16.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.262+0000 7fc0008b8640 1 -- 172.21.15.177:0/4222595451 wait complete. 2024-09-13T18:35:16.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 >> v1:172.21.15.177:6816/32265 conn(0x7f3ab004cb70 legacy=0x7f3ab004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 msgr2=0x7f3acc1394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 0x7f3acc1394e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009880 tx=0x7f3abc032000 comp rx=0 tx=0).stop 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 shutdown_connections 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3acc12e8d0 0x7f3acc11e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3acc098b70 0x7f3acc1394e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 --2- 172.21.15.177:0/3428368772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3acc093cc0 0x7f3acc138fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.263+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 >> 172.21.15.177:0/3428368772 conn(0x7f3acc096210 msgr2=0x7f3acc12daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.264+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 shutdown_connections 2024-09-13T18:35:16.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.264+0000 7f3aaffff640 1 -- 172.21.15.177:0/3428368772 wait complete. 2024-09-13T18:35:16.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2024-09-13T18:35:16.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-09-13T18:35:16.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.294+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff63805ed70 con 0x7ff638068260 2024-09-13T18:35:16.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.295+0000 7ff62dffb640 1 -- 172.21.15.177:0/2858015793 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff63401a030 con 0x7ff638068260 2024-09-13T18:35:16.298 INFO:teuthology.orchestra.run.smithi177.stdout:38654705667 2024-09-13T18:35:16.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 >> v1:172.21.15.177:6816/32265 conn(0x7ff61c04cb70 legacy=0x7ff61c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 msgr2=0x7ff638139050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 0x7ff638139050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff63400b810 tx=0x7ff6340091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 shutdown_connections 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff63812e8f0 0x7ff638139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff638068b70 0x7ff638139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 --2- 172.21.15.177:0/2858015793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff638068260 0x7ff638139050 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 >> 172.21.15.177:0/2858015793 conn(0x7ff638099d30 msgr2=0x7ff638092290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 shutdown_connections 2024-09-13T18:35:16.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.296+0000 7ff63fd5d640 1 -- 172.21.15.177:0/2858015793 wait complete. 2024-09-13T18:35:16.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.301+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f80ac06e490 con 0x7f80ac098a60 2024-09-13T18:35:16.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.301+0000 7f80a27fc640 1 -- 172.21.15.177:0/1959272774 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f80a803f560 con 0x7f80ac098a60 2024-09-13T18:35:16.304 INFO:teuthology.orchestra.run.smithi177.stdout:38654705667 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.302+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 >> v1:172.21.15.177:6816/32265 conn(0x7f809004cc30 legacy=0x7f809004f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.302+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 msgr2=0x7f80ac134ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.302+0000 7f80b479d640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac134ec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d890 tx=0x7f80a8008040 comp rx=0 tx=0).stop 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 shutdown_connections 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac128670 0x7f80ac087fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac127b70 0x7f80ac135400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 --2- 172.21.15.177:0/1959272774 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f80ac098a60 0x7f80ac134ec0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 >> 172.21.15.177:0/1959272774 conn(0x7f80ac096210 msgr2=0x7f80ac11e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:16.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 shutdown_connections 2024-09-13T18:35:16.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:16.303+0000 7f80b479d640 1 -- 172.21.15.177:0/1959272774 wait complete. 2024-09-13T18:35:16.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-09-13T18:35:16.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.1 2024-09-13T18:35:17.281 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:35:17.282 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:35:17.312 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:35:17.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:35:17.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.389+0000 7fcf1a99d640 1 Processor -- start 2024-09-13T18:35:17.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.389+0000 7fca9219d640 1 Processor -- start 2024-09-13T18:35:17.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.389+0000 7fcf1a99d640 1 -- start start 2024-09-13T18:35:17.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.389+0000 7fca9219d640 1 -- start start 2024-09-13T18:35:17.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf1a99d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 0x7fcf140928a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.396 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca9219d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fca8c068d40 0x7fca8c12f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.396 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca9219d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.396 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca9219d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c069370 con 0x7fca8c068260 2024-09-13T18:35:17.396 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca9219d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c069130 con 0x7fca8c068d40 2024-09-13T18:35:17.396 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf1a99d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf1412d880 0x7fcf1412fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.396 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf1a99d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf14093730 con 0x7fcf14092de0 2024-09-13T18:35:17.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca9219d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c069da0 con 0x7fca8c12f900 2024-09-13T18:35:17.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf1a99d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf14093d10 con 0x7fcf1412d880 2024-09-13T18:35:17.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf1a99d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14094830 con 0x7fcf140924d0 2024-09-13T18:35:17.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf18f3a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf1412d880 0x7fcf1412fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca8affd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fca8c068d40 0x7fca8c12f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca8b7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fca8c068260 0x7fca8c068630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47204/0 (socket says 172.21.15.177:47204) 2024-09-13T18:35:17.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fca8b7fe640 1 -- 172.21.15.177:0/4162155847 learned_addr learned my addr 172.21.15.177:0/4162155847 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:17.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf13fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 0x7fcf140928a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf18f3a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf1412d880 0x7fcf1412fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41690/0 (socket says 172.21.15.177:41690) 2024-09-13T18:35:17.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf18f3a640 1 -- 172.21.15.177:0/2925258285 learned_addr learned my addr 172.21.15.177:0/2925258285 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.390+0000 7fcf13fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 0x7fcf140928a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51354/0 (socket says 172.21.15.177:51354) 2024-09-13T18:35:17.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8a7fc640 1 -- 172.21.15.177:0/4162155847 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482230278 0 0) 0x7fca8c069370 con 0x7fca8c068260 2024-09-13T18:35:17.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8a7fc640 1 -- 172.21.15.177:0/4162155847 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca740034e0 con 0x7fca8c068260 2024-09-13T18:35:17.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8affd640 1 -- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 msgr2=0x7fca8c131ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:17.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fcf18f3a640 1 -- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 msgr2=0x7fcf140928a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fcf18f3a640 1 --2- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 0x7fcf140928a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fcf18f3a640 1 -- 172.21.15.177:0/2925258285 >> v1:172.21.15.177:6789/0 conn(0x7fcf14092de0 legacy=0x7fcf1412c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fcf18f3a640 1 -- 172.21.15.177:0/2925258285 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14094570 con 0x7fcf1412d880 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fcf13fff640 1 --2- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 0x7fcf140928a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8affd640 1 --2- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c131ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8affd640 1 -- 172.21.15.177:0/4162155847 >> v1:172.21.15.177:6789/0 conn(0x7fca8c068260 legacy=0x7fca8c068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8affd640 1 -- 172.21.15.177:0/4162155847 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0940f0 con 0x7fca8c068d40 2024-09-13T18:35:17.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fca8bfff640 1 --2- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.391+0000 7fcf18f3a640 1 --2- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf1412d880 0x7fcf1412fc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00f490 tx=0x7fcf0c00dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30245124803e831 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf12ffd640 1 -- 172.21.15.177:0/2925258285 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00d2e0 con 0x7fcf1412d880 2024-09-13T18:35:17.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf12ffd640 1 -- 172.21.15.177:0/2925258285 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c00d440 con 0x7fcf1412d880 2024-09-13T18:35:17.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf12ffd640 1 -- 172.21.15.177:0/2925258285 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00b810 con 0x7fcf1412d880 2024-09-13T18:35:17.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fca8affd640 1 --2- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068d40 0x7fca8c12f3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca80009850 tx=0x7fca800090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e749a6a48b94186e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fca8a7fc640 1 -- 172.21.15.177:0/4162155847 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8002ac70 con 0x7fca8c068d40 2024-09-13T18:35:17.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fca8a7fc640 1 -- 172.21.15.177:0/4162155847 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8002add0 con 0x7fca8c068d40 2024-09-13T18:35:17.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fca8a7fc640 1 -- 172.21.15.177:0/4162155847 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80032670 con 0x7fca8c068d40 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf1a99d640 1 -- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf1412d880 msgr2=0x7fcf1412fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf1412d880 0x7fcf1412fc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00f490 tx=0x7fcf0c00dc50 comp rx=0 tx=0).stop 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf1a99d640 1 -- 172.21.15.177:0/2925258285 shutdown_connections 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf1412d880 0x7fcf1412fc60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/2925258285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf140924d0 0x7fcf140928a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.392+0000 7fcf1a99d640 1 -- 172.21.15.177:0/2925258285 >> 172.21.15.177:0/2925258285 conn(0x7fcf14097ae0 msgr2=0x7fcf14097ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fcf1a99d640 1 -- 172.21.15.177:0/2925258285 shutdown_connections 2024-09-13T18:35:17.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fcf1a99d640 1 -- 172.21.15.177:0/2925258285 wait complete. 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 -- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068d40 msgr2=0x7fca8c12f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 --2- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068d40 0x7fca8c12f3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca80009850 tx=0x7fca800090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 -- 172.21.15.177:0/4162155847 shutdown_connections 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 --2- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 --2- 172.21.15.177:0/4162155847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068d40 0x7fca8c12f3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 -- 172.21.15.177:0/4162155847 >> 172.21.15.177:0/4162155847 conn(0x7fca8c099d30 msgr2=0x7fca8c09a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 -- 172.21.15.177:0/4162155847 shutdown_connections 2024-09-13T18:35:17.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fca9219d640 1 -- 172.21.15.177:0/4162155847 wait complete. 2024-09-13T18:35:17.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.393+0000 7fcf1a99d640 1 Processor -- start 2024-09-13T18:35:17.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 -- start start 2024-09-13T18:35:17.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fca9219d640 1 Processor -- start 2024-09-13T18:35:17.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fca9219d640 1 -- start start 2024-09-13T18:35:17.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fca9219d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 0x7fca8c08ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 0x7fcf14134d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fca9219d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fca8c068d40 0x7fca8c08e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fca9219d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c08e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf14092de0 0x7fcf14135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf1412d880 0x7fcf141357b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1412cb30 con 0x7fcf140924d0 2024-09-13T18:35:17.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1412c420 con 0x7fcf14092de0 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf1a99d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1412d430 con 0x7fcf1412d880 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf13fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 0x7fcf14134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.394+0000 7fcf137fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf14092de0 0x7fcf14135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf13fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 0x7fcf14134d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41706/0 (socket says 172.21.15.177:41706) 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf13fff640 1 -- 172.21.15.177:0/1972880445 learned_addr learned my addr 172.21.15.177:0/1972880445 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf18f3a640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf1412d880 0x7fcf141357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf137fe640 1 -- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf1412d880 msgr2=0x7fcf141357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf137fe640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf1412d880 0x7fcf141357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf137fe640 1 -- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 msgr2=0x7fcf14134d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf137fe640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 0x7fcf14134d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c08e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 0x7fca8c08ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 0x7fca8c08ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41708/0 (socket says 172.21.15.177:41708) 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 -- 172.21.15.177:0/997069332 learned_addr learned my addr 172.21.15.177:0/997069332 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c08fc50 con 0x7fca8c068260 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c122730 con 0x7fca8c12f900 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c122420 con 0x7fca8c068d40 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 -- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fca8c068d40 msgr2=0x7fca8c08e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8affd640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fca8c068d40 0x7fca8c08e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fca8c068d40 0x7fca8c08e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 -- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 msgr2=0x7fca8c08e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c08e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf137fe640 1 -- 172.21.15.177:0/1972880445 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009390 con 0x7fcf14092de0 2024-09-13T18:35:17.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca84009a20 con 0x7fca8c068260 2024-09-13T18:35:17.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf13fff640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 0x7fcf14134d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8bfff640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c08e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8affd640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fca8c068d40 0x7fca8c08e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:35:17.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fcf137fe640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf14092de0 0x7fcf14135270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c4b0 tx=0x7fcf08007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.395+0000 7fca8b7fe640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 0x7fca8c08ddc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca7c012fd0 tx=0x7fca7c007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08011590 con 0x7fcf14092de0 2024-09-13T18:35:17.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c014c10 con 0x7fca8c068260 2024-09-13T18:35:17.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf080116f0 con 0x7fcf14092de0 2024-09-13T18:35:17.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08011af0 con 0x7fcf14092de0 2024-09-13T18:35:17.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c00f050 con 0x7fcf14092de0 2024-09-13T18:35:17.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1412ad50 con 0x7fcf14092de0 2024-09-13T18:35:17.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca80009390 con 0x7fca8c068260 2024-09-13T18:35:17.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c014d70 con 0x7fca8c068260 2024-09-13T18:35:17.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0073c0 con 0x7fca8c068260 2024-09-13T18:35:17.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.396+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c12f550 con 0x7fca8c068260 2024-09-13T18:35:17.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.397+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1405ed70 con 0x7fcf14092de0 2024-09-13T18:35:17.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.397+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fcf0801b3b0 con 0x7fcf14092de0 2024-09-13T18:35:17.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.398+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c05ed70 con 0x7fca8c068260 2024-09-13T18:35:17.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.398+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fca7c0047a0 con 0x7fca8c068260 2024-09-13T18:35:17.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.398+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fcf08062680 con 0x7fcf14092de0 2024-09-13T18:35:17.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.398+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fca7c073980 con 0x7fca8c068260 2024-09-13T18:35:17.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.402+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fca7c04eac0 con 0x7fca8c068260 2024-09-13T18:35:17.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.404+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fcf0803d760 con 0x7fcf14092de0 2024-09-13T18:35:17.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.428+0000 7f2c5659d640 1 Processor -- start 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.428+0000 7f2c5659d640 1 -- start start 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c5659d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c50068d40 0x7f2c5012f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c5659d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c5012f900 0x7f2c50131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c5659d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c50069370 con 0x7f2c50068260 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c5659d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50069130 con 0x7f2c50068d40 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c5659d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50069da0 con 0x7f2c5012f900 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c50068d40 0x7f2c5012f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c54b3a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c5012f900 0x7f2c50131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c4ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2c50068260 0x7f2c50068630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47226/0 (socket says 172.21.15.177:47226) 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c4ffff640 1 -- 172.21.15.177:0/853412869 learned_addr learned my addr 172.21.15.177:0/853412869 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c54b3a640 1 -- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c50068d40 msgr2=0x7f2c5012f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c54b3a640 1 --2- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c50068d40 0x7f2c5012f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c4effd640 1 -- 172.21.15.177:0/853412869 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386667902 0 0) 0x7f2c50069370 con 0x7f2c50068260 2024-09-13T18:35:17.434 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c54b3a640 1 -- 172.21.15.177:0/853412869 >> v1:172.21.15.177:6789/0 conn(0x7f2c50068260 legacy=0x7f2c50068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.434 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.429+0000 7f2c54b3a640 1 -- 172.21.15.177:0/853412869 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500940f0 con 0x7f2c5012f900 2024-09-13T18:35:17.434 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.430+0000 7f2c4f7fe640 1 --2- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c50068d40 0x7f2c5012f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c54b3a640 1 --2- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c5012f900 0x7f2c50131ce0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009ec0 tx=0x7f2c48009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8693776bdcbedd28 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c4effd640 1 -- 172.21.15.177:0/853412869 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48031400 con 0x7f2c5012f900 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c4effd640 1 -- 172.21.15.177:0/853412869 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4802d050 con 0x7f2c5012f900 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c4effd640 1 -- 172.21.15.177:0/853412869 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48031790 con 0x7f2c5012f900 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c5659d640 1 -- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c5012f900 msgr2=0x7f2c50131ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c5659d640 1 --2- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c5012f900 0x7f2c50131ce0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009ec0 tx=0x7f2c48009780 comp rx=0 tx=0).stop 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c5659d640 1 -- 172.21.15.177:0/853412869 shutdown_connections 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c5659d640 1 --2- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c50068d40 0x7f2c5012f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c5659d640 1 --2- 172.21.15.177:0/853412869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c5012f900 0x7f2c50131ce0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.431+0000 7f2c5659d640 1 -- 172.21.15.177:0/853412869 >> 172.21.15.177:0/853412869 conn(0x7f2c50099d30 msgr2=0x7f2c5009a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.432+0000 7f2c5659d640 1 -- 172.21.15.177:0/853412869 shutdown_connections 2024-09-13T18:35:17.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.432+0000 7f2c5659d640 1 -- 172.21.15.177:0/853412869 wait complete. 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.432+0000 7f2c5659d640 1 Processor -- start 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.432+0000 7f2c5659d640 1 -- start start 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c5659d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 0x7f2c5012f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c5659d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c50068d40 0x7f2c5008daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c5659d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c5012f900 0x7f2c5008e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c5659d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5009c7b0 con 0x7f2c50068260 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 0x7f2c5012f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c5659d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5008fde0 con 0x7f2c5012f900 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c5659d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50090ee0 con 0x7f2c50068d40 2024-09-13T18:35:17.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 0x7f2c5012f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41720/0 (socket says 172.21.15.177:41720) 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c4ffff640 1 -- 172.21.15.177:0/1719102095 learned_addr learned my addr 172.21.15.177:0/1719102095 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c4f7fe640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c50068d40 0x7f2c5008daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c54b3a640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c5012f900 0x7f2c5008e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.433+0000 7f2c4ffff640 1 -- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c50068d40 msgr2=0x7f2c5008daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c4ffff640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c50068d40 0x7f2c5008daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c4ffff640 1 -- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c5012f900 msgr2=0x7f2c5008e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c4ffff640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c5012f900 0x7f2c5008e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c4ffff640 1 -- 172.21.15.177:0/1719102095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44009390 con 0x7f2c50068260 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c4f7fe640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c50068d40 0x7f2c5008daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c54b3a640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c5012f900 0x7f2c5008e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c4ffff640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 0x7f2c5012f370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c5e0 tx=0x7f2c40007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40005aa0 con 0x7f2c50068260 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c40005c00 con 0x7f2c50068260 2024-09-13T18:35:17.439 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400070f0 con 0x7f2c50068260 2024-09-13T18:35:17.440 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c48009a20 con 0x7f2c50068260 2024-09-13T18:35:17.440 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.434+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5008e710 con 0x7f2c50068260 2024-09-13T18:35:17.440 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.436+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5005ed70 con 0x7f2c50068260 2024-09-13T18:35:17.440 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.436+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f2c40007520 con 0x7f2c50068260 2024-09-13T18:35:17.440 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.436+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f2c40062be0 con 0x7f2c50068260 2024-09-13T18:35:17.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.440+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f2c4003de60 con 0x7f2c50068260 2024-09-13T18:35:17.445 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 Processor -- start 2024-09-13T18:35:17.445 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 -- start start 2024-09-13T18:35:17.446 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b812b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.446 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b812bde0 0x7f99b812e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.446 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b806b8d0 con 0x7f99b8069c70 2024-09-13T18:35:17.446 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b806ab40 con 0x7f99b812bde0 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bf49f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b806eab0 con 0x7f99b806a750 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.442+0000 7f99bca3a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b812b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bca3a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b812b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51394/0 (socket says 172.21.15.177:51394) 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bca3a640 1 -- 172.21.15.177:0/1247195702 learned_addr learned my addr 172.21.15.177:0/1247195702 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bd23b640 1 --1- 172.21.15.177:0/1247195702 >> v1:172.21.15.177:6789/0 conn(0x7f99b8069c70 0x7f99b806a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47240/0 (socket says 172.21.15.177:47240) 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bda3c640 1 --2- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b812bde0 0x7f99b812e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bca3a640 1 -- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b812bde0 msgr2=0x7f99b812e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bca3a640 1 --2- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b812bde0 0x7f99b812e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99affff640 1 -- 172.21.15.177:0/1247195702 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60005802 0 0) 0x7f99b806b8d0 con 0x7f99b8069c70 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bca3a640 1 -- 172.21.15.177:0/1247195702 >> v1:172.21.15.177:6789/0 conn(0x7f99b8069c70 legacy=0x7f99b806a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.443+0000 7f99bca3a640 1 -- 172.21.15.177:0/1247195702 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8099050 con 0x7f99b806a750 2024-09-13T18:35:17.449 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.444+0000 7f99bca3a640 1 --2- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b812b8a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009940 tx=0x7f99a80091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b0a3d508e7af3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.449 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.444+0000 7f99affff640 1 -- 172.21.15.177:0/1247195702 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a802ad00 con 0x7f99b806a750 2024-09-13T18:35:17.449 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.444+0000 7f99affff640 1 -- 172.21.15.177:0/1247195702 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a802ae60 con 0x7f99b806a750 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.444+0000 7f99affff640 1 -- 172.21.15.177:0/1247195702 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8032670 con 0x7f99b806a750 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 -- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 msgr2=0x7f99b812b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 --2- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b812b8a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009940 tx=0x7f99a80091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 -- 172.21.15.177:0/1247195702 shutdown_connections 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 --2- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b812bde0 0x7f99b812e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 --2- 172.21.15.177:0/1247195702 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b812b8a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 -- 172.21.15.177:0/1247195702 >> 172.21.15.177:0/1247195702 conn(0x7f99b8096210 msgr2=0x7f99b8096610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 -- 172.21.15.177:0/1247195702 shutdown_connections 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.445+0000 7f99bf49f640 1 -- 172.21.15.177:0/1247195702 wait complete. 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.446+0000 7f99bf49f640 1 Processor -- start 2024-09-13T18:35:17.450 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.446+0000 7f99bf49f640 1 -- start start 2024-09-13T18:35:17.452 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bf49f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 0x7f99b81238a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.452 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bf49f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b8123de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.452 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bf49f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99b812bde0 0x7f99b8124320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bd23b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 0x7f99b81238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bf49f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b81286f0 con 0x7f99b812bde0 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bf49f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8126c20 con 0x7f99b8069c70 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bf49f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8126d90 con 0x7f99b806a750 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bd23b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 0x7f99b81238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41744/0 (socket says 172.21.15.177:41744) 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bd23b640 1 -- 172.21.15.177:0/2875907280 learned_addr learned my addr 172.21.15.177:0/2875907280 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bca3a640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b8123de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.447+0000 7f99bda3c640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99b812bde0 0x7f99b8124320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bd23b640 1 -- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 msgr2=0x7f99b8123de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bd23b640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b8123de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bd23b640 1 -- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99b812bde0 msgr2=0x7f99b8124320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bd23b640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99b812bde0 0x7f99b8124320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bd23b640 1 -- 172.21.15.177:0/2875907280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009450 con 0x7f99b8069c70 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bda3c640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99b812bde0 0x7f99b8124320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bd23b640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 0x7f99b81238a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c610 tx=0x7f99b4007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4007300 con 0x7f99b8069c70 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4007480 con 0x7f99b8069c70 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40049f0 con 0x7f99b8069c70 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.448+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b8126730 con 0x7f99b8069c70 2024-09-13T18:35:17.454 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.449+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b81272b0 con 0x7f99b8069c70 2024-09-13T18:35:17.455 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.449+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f99b4012440 con 0x7f99b8069c70 2024-09-13T18:35:17.455 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.450+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f99b4067d10 con 0x7f99b8069c70 2024-09-13T18:35:17.455 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.450+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b805ed70 con 0x7f99b8069c70 2024-09-13T18:35:17.457 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.454+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f99b4018070 con 0x7f99b8069c70 2024-09-13T18:35:17.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.573+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fca8c093420 con 0x7fca8c068260 2024-09-13T18:35:17.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.573+0000 7fca737fe640 1 -- 172.21.15.177:0/997069332 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca7c0522d0 con 0x7fca8c068260 2024-09-13T18:35:17.576 INFO:teuthology.orchestra.run.smithi177.stdout:38654705669 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 >> v1:172.21.15.177:6816/32265 conn(0x7fca7804cba0 legacy=0x7fca7804f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 msgr2=0x7fca8c08ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 0x7fca8c08ddc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca7c012fd0 tx=0x7fca7c007e40 comp rx=0 tx=0).stop 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 shutdown_connections 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca8c12f900 0x7fca8c08e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fca8c068d40 0x7fca8c08e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 --2- 172.21.15.177:0/997069332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca8c068260 0x7fca8c08ddc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 >> 172.21.15.177:0/997069332 conn(0x7fca8c099d30 msgr2=0x7fca8c096730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 shutdown_connections 2024-09-13T18:35:17.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.575+0000 7fca9219d640 1 -- 172.21.15.177:0/997069332 wait complete. 2024-09-13T18:35:17.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2024-09-13T18:35:17.589 DEBUG:teuthology.parallel:result is None 2024-09-13T18:35:17.601 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.596+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcf14120480 con 0x7fcf14092de0 2024-09-13T18:35:17.601 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.597+0000 7fcefbfff640 1 -- 172.21.15.177:0/1972880445 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf08017070 con 0x7fcf14092de0 2024-09-13T18:35:17.601 INFO:teuthology.orchestra.run.smithi177.stdout:38654705669 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 >> v1:172.21.15.177:6816/32265 conn(0x7fcf0004cb70 legacy=0x7fcf0004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf14092de0 msgr2=0x7fcf14135270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf14092de0 0x7fcf14135270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c4b0 tx=0x7fcf08007e30 comp rx=0 tx=0).stop 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 shutdown_connections 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcf1412d880 0x7fcf141357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf14092de0 0x7fcf14135270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 --2- 172.21.15.177:0/1972880445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf140924d0 0x7fcf14134d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 >> 172.21.15.177:0/1972880445 conn(0x7fcf14097ae0 msgr2=0x7fcf14099cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 shutdown_connections 2024-09-13T18:35:17.603 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.598+0000 7fcf1a99d640 1 -- 172.21.15.177:0/1972880445 wait complete. 2024-09-13T18:35:17.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-09-13T18:35:17.612 DEBUG:teuthology.parallel:result is None 2024-09-13T18:35:17.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.610+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2c50093080 con 0x7f2c50068260 2024-09-13T18:35:17.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.610+0000 7f2c37fff640 1 -- 172.21.15.177:0/1719102095 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c40041670 con 0x7f2c50068260 2024-09-13T18:35:17.613 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:17.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 >> v1:172.21.15.177:6816/32265 conn(0x7f2c3c04cba0 legacy=0x7f2c3c04f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 msgr2=0x7f2c5012f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 0x7f2c5012f370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c5e0 tx=0x7f2c40007d00 comp rx=0 tx=0).stop 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 shutdown_connections 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c5012f900 0x7f2c5008e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2c50068d40 0x7f2c5008daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 --2- 172.21.15.177:0/1719102095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c50068260 0x7f2c5012f370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.611+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 >> 172.21.15.177:0/1719102095 conn(0x7f2c50099d30 msgr2=0x7f2c5009c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.612+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 shutdown_connections 2024-09-13T18:35:17.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.612+0000 7f2c5659d640 1 -- 172.21.15.177:0/1719102095 wait complete. 2024-09-13T18:35:17.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2024-09-13T18:35:17.626 DEBUG:teuthology.parallel:result is None 2024-09-13T18:35:17.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.630+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f99b806dbb0 con 0x7f99b8069c70 2024-09-13T18:35:17.633 INFO:teuthology.orchestra.run.smithi177.stdout:38654705668 2024-09-13T18:35:17.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.630+0000 7f99ad7fa640 1 -- 172.21.15.177:0/2875907280 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f99b4068060 con 0x7f99b8069c70 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.631+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 >> v1:172.21.15.177:6816/32265 conn(0x7f999c04cb70 legacy=0x7f999c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.631+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 msgr2=0x7f99b81238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.631+0000 7f99bf49f640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 0x7f99b81238a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c610 tx=0x7f99b4007d00 comp rx=0 tx=0).stop 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 shutdown_connections 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99b812bde0 0x7f99b8124320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f99b806a750 0x7f99b8123de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 --2- 172.21.15.177:0/2875907280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99b8069c70 0x7f99b81238a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 >> 172.21.15.177:0/2875907280 conn(0x7f99b8096210 msgr2=0x7f99b812aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 shutdown_connections 2024-09-13T18:35:17.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.632+0000 7f99bf49f640 1 -- 172.21.15.177:0/2875907280 wait complete. 2024-09-13T18:35:17.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.1 2024-09-13T18:35:17.645 DEBUG:teuthology.parallel:result is None 2024-09-13T18:35:17.645 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:35:17.645 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:35:17.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.776+0000 7f6fde51d640 1 Processor -- start 2024-09-13T18:35:17.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.776+0000 7f6fde51d640 1 -- start start 2024-09-13T18:35:17.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fde51d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd81263f0 0x7f6fd81267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fde51d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 0x7f6fd8090350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fde51d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6fd8092f10 0x7f6fd8094800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fde51d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd80916f0 con 0x7f6fd8092f10 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fde51d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80952c0 con 0x7f6fd81263f0 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fde51d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8094d40 con 0x7f6fd8126d00 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 0x7f6fd8090350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd7fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd81263f0 0x7f6fd81267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fdcaba640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6fd8092f10 0x7f6fd8094800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 0x7f6fd8090350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51408/0 (socket says 172.21.15.177:51408) 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd7fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd81263f0 0x7f6fd81267c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:41754/0 (socket says 172.21.15.177:41754) 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3765185741 learned_addr learned my addr 172.21.15.177:0/3765185741 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd81263f0 msgr2=0x7f6fd81267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 --2- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd81263f0 0x7f6fd81267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3765185741 >> v2:172.21.15.177:3300/0 conn(0x7f6fd8092f10 msgr2=0x7f6fd8094800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:17.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 --2- 172.21.15.177:0/3765185741 >> v2:172.21.15.177:3300/0 conn(0x7f6fd8092f10 0x7f6fd8094800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3765185741 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8094f00 con 0x7f6fd8126d00 2024-09-13T18:35:17.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.777+0000 7f6fdcaba640 1 --2- 172.21.15.177:0/3765185741 >> v2:172.21.15.177:3300/0 conn(0x7f6fd8092f10 0x7f6fd8094800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fd77fe640 1 --2- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 0x7f6fd8090350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009910 tx=0x7f6fcc0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797b9e5bdfc69f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fd6ffd640 1 -- 172.21.15.177:0/3765185741 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc02ac90 con 0x7f6fd8126d00 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fd6ffd640 1 -- 172.21.15.177:0/3765185741 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc02adf0 con 0x7f6fd8126d00 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fd6ffd640 1 -- 172.21.15.177:0/3765185741 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc032670 con 0x7f6fd8126d00 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 -- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 msgr2=0x7f6fd8090350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 0x7f6fd8090350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009910 tx=0x7f6fcc0091d0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 -- 172.21.15.177:0/3765185741 shutdown_connections 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3765185741 >> v2:172.21.15.177:3300/0 conn(0x7f6fd8092f10 0x7f6fd8094800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8126d00 0x7f6fd8090350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3765185741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd81263f0 0x7f6fd81267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 -- 172.21.15.177:0/3765185741 >> 172.21.15.177:0/3765185741 conn(0x7f6fd8099a40 msgr2=0x7f6fd80958f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.778+0000 7f6fde51d640 1 -- 172.21.15.177:0/3765185741 shutdown_connections 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.779+0000 7f6fde51d640 1 -- 172.21.15.177:0/3765185741 wait complete. 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.779+0000 7f6fde51d640 1 Processor -- start 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.779+0000 7f6fde51d640 1 -- start start 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fde51d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 0x7f6fd8139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fde51d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fd81263f0 0x7f6fd81397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fde51d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd8126d00 0x7f6fd8139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fde51d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8121110 con 0x7f6fd81263f0 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fde51d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd81214e0 con 0x7f6fd8126d00 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fde51d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd806ebc0 con 0x7f6fd8092f10 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd7fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 0x7f6fd8139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fdcaba640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd8126d00 0x7f6fd8139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fd81263f0 0x7f6fd81397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd7fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 0x7f6fd8139270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51424/0 (socket says 172.21.15.177:51424) 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd7fff640 1 -- 172.21.15.177:0/3571722032 learned_addr learned my addr 172.21.15.177:0/3571722032 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 msgr2=0x7f6fd8139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 0x7f6fd8139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd8126d00 msgr2=0x7f6fd8139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd8126d00 0x7f6fd8139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 -- 172.21.15.177:0/3571722032 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0009a20 con 0x7f6fd81263f0 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fdcaba640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd8126d00 0x7f6fd8139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd7fff640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 0x7f6fd8139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fd77fe640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fd81263f0 0x7f6fd81397b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc031000 tx=0x7f6fcc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.780+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc03d650 con 0x7f6fd81263f0 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.781+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc03d7b0 con 0x7f6fd81263f0 2024-09-13T18:35:17.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.781+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc009450 con 0x7f6fd81263f0 2024-09-13T18:35:17.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.781+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc03db70 con 0x7f6fd81263f0 2024-09-13T18:35:17.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.781+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd811ded0 con 0x7f6fd81263f0 2024-09-13T18:35:17.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.782+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f6fcc039040 con 0x7f6fd81263f0 2024-09-13T18:35:17.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.782+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd805ed70 con 0x7f6fd81263f0 2024-09-13T18:35:17.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.782+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f6fcc0824f0 con 0x7f6fd81263f0 2024-09-13T18:35:17.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.785+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f6fcc05d630 con 0x7f6fd81263f0 2024-09-13T18:35:17.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.914+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 --> v1:172.21.15.177:6816/32265 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fd8120890 con 0x7f6fc404cb70 2024-09-13T18:35:17.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.915+0000 7f6fbffff640 1 -- 172.21.15.177:0/3571722032 <== mgr.4108 v1:172.21.15.177:6816/32265 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34995 (unknown 2965378022 0 1909064646) 0x7f6fc800bd20 con 0x7f6fc404cb70 2024-09-13T18:35:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:17.919 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 >> v1:172.21.15.177:6816/32265 conn(0x7f6fc404cb70 legacy=0x7f6fc404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fd81263f0 msgr2=0x7f6fd81397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fd81263f0 0x7f6fd81397b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc031000 tx=0x7f6fcc032000 comp rx=0 tx=0).stop 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 shutdown_connections 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fd8126d00 0x7f6fd8139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fd81263f0 0x7f6fd81397b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 --2- 172.21.15.177:0/3571722032 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6fd8092f10 0x7f6fd8139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 >> 172.21.15.177:0/3571722032 conn(0x7f6fd8099a40 msgr2=0x7f6fd806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 shutdown_connections 2024-09-13T18:35:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:17.917+0000 7f6fde51d640 1 -- 172.21.15.177:0/3571722032 wait complete. 2024-09-13T18:35:17.930 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T18:35:16.613181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191140,"kb_used_data":1692,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296220,"statfs":{"total":386547056640,"available":385327329280,"internally_reserved":0,"allocated":1732608,"data_stored":1323901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.893851"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.735221+0000","last_change":"2024-09-13T18:35:13.735440+0000","last_active":"2024-09-13T18:35:13.735221+0000","last_peered":"2024-09-13T18:35:13.735221+0000","last_clean":"2024-09-13T18:35:13.735221+0000","last_became_active":"2024-09-13T18:35:11.730650+0000","last_became_peered":"2024-09-13T18:35:11.730650+0000","last_unstale":"2024-09-13T18:35:13.735221+0000","last_undegraded":"2024-09-13T18:35:13.735221+0000","last_fullsized":"2024-09-13T18:35:13.735221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:35.164028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734812+0000","last_change":"2024-09-13T18:35:13.735179+0000","last_active":"2024-09-13T18:35:13.734812+0000","last_peered":"2024-09-13T18:35:13.734812+0000","last_clean":"2024-09-13T18:35:13.734812+0000","last_became_active":"2024-09-13T18:35:11.730100+0000","last_became_peered":"2024-09-13T18:35:11.730100+0000","last_unstale":"2024-09-13T18:35:13.734812+0000","last_undegraded":"2024-09-13T18:35:13.734812+0000","last_fullsized":"2024-09-13T18:35:13.734812+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:25.061944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.735530+0000","last_change":"2024-09-13T18:35:13.735829+0000","last_active":"2024-09-13T18:35:13.735530+0000","last_peered":"2024-09-13T18:35:13.735530+0000","last_clean":"2024-09-13T18:35:13.735530+0000","last_became_active":"2024-09-13T18:35:11.730242+0000","last_became_peered":"2024-09-13T18:35:11.730242+0000","last_unstale":"2024-09-13T18:35:13.735530+0000","last_undegraded":"2024-09-13T18:35:13.735530+0000","last_fullsized":"2024-09-13T18:35:13.735530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:01:03.137288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734864+0000","last_change":"2024-09-13T18:35:13.735117+0000","last_active":"2024-09-13T18:35:13.734864+0000","last_peered":"2024-09-13T18:35:13.734864+0000","last_clean":"2024-09-13T18:35:13.734864+0000","last_became_active":"2024-09-13T18:35:11.730015+0000","last_became_peered":"2024-09-13T18:35:11.730015+0000","last_unstale":"2024-09-13T18:35:13.734864+0000","last_undegraded":"2024-09-13T18:35:13.734864+0000","last_fullsized":"2024-09-13T18:35:13.734864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:43.650449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"17'2","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.735587+0000","last_change":"2024-09-13T18:35:13.735587+0000","last_active":"2024-09-13T18:35:13.735587+0000","last_peered":"2024-09-13T18:35:13.735587+0000","last_clean":"2024-09-13T18:35:13.735587+0000","last_became_active":"2024-09-13T18:35:11.733034+0000","last_became_peered":"2024-09-13T18:35:11.733034+0000","last_unstale":"2024-09-13T18:35:13.735587+0000","last_undegraded":"2024-09-13T18:35:13.735587+0000","last_fullsized":"2024-09-13T18:35:13.735587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:01:22.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:14.461023+0000","last_change":"2024-09-13T18:35:14.461158+0000","last_active":"2024-09-13T18:35:14.461023+0000","last_peered":"2024-09-13T18:35:14.461023+0000","last_clean":"2024-09-13T18:35:14.461023+0000","last_became_active":"2024-09-13T18:35:11.729626+0000","last_became_peered":"2024-09-13T18:35:11.729626+0000","last_unstale":"2024-09-13T18:35:14.461023+0000","last_undegraded":"2024-09-13T18:35:14.461023+0000","last_fullsized":"2024-09-13T18:35:14.461023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:10.111028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734705+0000","last_change":"2024-09-13T18:35:13.735139+0000","last_active":"2024-09-13T18:35:13.734705+0000","last_peered":"2024-09-13T18:35:13.734705+0000","last_clean":"2024-09-13T18:35:13.734705+0000","last_became_active":"2024-09-13T18:35:11.733025+0000","last_became_peered":"2024-09-13T18:35:11.733025+0000","last_unstale":"2024-09-13T18:35:13.734705+0000","last_undegraded":"2024-09-13T18:35:13.734705+0000","last_fullsized":"2024-09-13T18:35:13.734705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.647913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"15'1","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734760+0000","last_change":"2024-09-13T18:35:13.735186+0000","last_active":"2024-09-13T18:35:13.734760+0000","last_peered":"2024-09-13T18:35:13.734760+0000","last_clean":"2024-09-13T18:35:13.734760+0000","last_became_active":"2024-09-13T18:35:11.727196+0000","last_became_peered":"2024-09-13T18:35:11.727196+0000","last_unstale":"2024-09-13T18:35:13.734760+0000","last_undegraded":"2024-09-13T18:35:13.734760+0000","last_fullsized":"2024-09-13T18:35:13.734760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:18.070546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"13'52","reported_seq":80,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T18:35:12.730721+0000","last_change":"2024-09-13T18:35:07.177729+0000","last_active":"2024-09-13T18:35:12.730721+0000","last_peered":"2024-09-13T18:35:12.730721+0000","last_clean":"2024-09-13T18:35:12.730721+0000","last_became_active":"2024-09-13T18:35:07.176786+0000","last_became_peered":"2024-09-13T18:35:07.176786+0000","last_unstale":"2024-09-13T18:35:12.730721+0000","last_undegraded":"2024-09-13T18:35:12.730721+0000","last_fullsized":"2024-09-13T18:35:12.730721+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:02.631221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":143360,"data_stored":37617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":131072,"data_stored":34271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":737280,"data_stored":627985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298064,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073776,"statfs":{"total":96636764160,"available":96331546624,"internally_reserved":0,"allocated":720896,"data_stored":624028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:35:17.931 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:35:18.025 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.020+0000 7fa616a20640 1 Processor -- start 2024-09-13T18:35:18.025 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 -- start start 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa610091830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa610091d70 0x7fa61012af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa61012b4c0 0x7fa61012d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61008f0c0 con 0x7fa610091d70 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa610092a30 con 0x7fa61012b4c0 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa616a20640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa610069380 con 0x7fa610091460 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa610091d70 0x7fa61012af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa614fbd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa61012b4c0 0x7fa61012d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa610091830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa610091d70 0x7fa61012af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41746/0 (socket says 172.21.15.177:41746) 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 -- 172.21.15.177:0/3788342254 learned_addr learned my addr 172.21.15.177:0/3788342254 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 -- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 msgr2=0x7fa610091830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa610091830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.026 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 -- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa61012b4c0 msgr2=0x7fa61012d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa61012b4c0 0x7fa61012d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60f7fe640 1 -- 172.21.15.177:0/3788342254 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610092c60 con 0x7fa610091d70 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa614fbd640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa61012b4c0 0x7fa61012d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.021+0000 7fa60ffff640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa610091830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.022+0000 7fa60f7fe640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa610091d70 0x7fa61012af80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa60400bbc0 tx=0x7fa60400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f0717d4397d21c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.022+0000 7fa60effd640 1 -- 172.21.15.177:0/3788342254 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040324f0 con 0x7fa610091d70 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.022+0000 7fa60effd640 1 -- 172.21.15.177:0/3788342254 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604032650 con 0x7fa610091d70 2024-09-13T18:35:18.027 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.022+0000 7fa60effd640 1 -- 172.21.15.177:0/3788342254 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040329f0 con 0x7fa610091d70 2024-09-13T18:35:18.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 -- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa610091d70 msgr2=0x7fa61012af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa610091d70 0x7fa61012af80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa60400bbc0 tx=0x7fa60400b460 comp rx=0 tx=0).stop 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 -- 172.21.15.177:0/3788342254 shutdown_connections 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa61012b4c0 0x7fa61012d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa610091d70 0x7fa61012af80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 --2- 172.21.15.177:0/3788342254 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa610091830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 -- 172.21.15.177:0/3788342254 >> 172.21.15.177:0/3788342254 conn(0x7fa610099a40 msgr2=0x7fa6100958f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 -- 172.21.15.177:0/3788342254 shutdown_connections 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.023+0000 7fa616a20640 1 -- 172.21.15.177:0/3788342254 wait complete. 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 Processor -- start 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 -- start start 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa61008bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa610091d70 0x7fa61008c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa61012b4c0 0x7fa61008c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61008d670 con 0x7fa61012b4c0 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61011e0c0 con 0x7fa610091d70 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa616a20640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa61008dec0 con 0x7fa610091460 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa61008bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa60f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa610091d70 0x7fa61008c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa61008bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:51434/0 (socket says 172.21.15.177:51434) 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa60ffff640 1 -- 172.21.15.177:0/229930781 learned_addr learned my addr 172.21.15.177:0/229930781 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.024+0000 7fa614fbd640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa61012b4c0 0x7fa61008c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa60ffff640 1 -- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa610091d70 msgr2=0x7fa61008c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa60ffff640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa610091d70 0x7fa61008c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa60ffff640 1 -- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa61012b4c0 msgr2=0x7fa61008c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa60ffff640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa61012b4c0 0x7fa61008c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.031 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa60ffff640 1 -- 172.21.15.177:0/229930781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa600009390 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa60ffff640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa61008bd80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa60000ec80 tx=0x7fa6000090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000eec0 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60000cb00 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000ce80 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa608009a20 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.025+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60400b700 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.026+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61008e820 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.026+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fa600013680 con 0x7fa610091460 2024-09-13T18:35:18.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.027+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fa6000634c0 con 0x7fa610091460 2024-09-13T18:35:18.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.030+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fa60003e600 con 0x7fa610091460 2024-09-13T18:35:18.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.156+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 --> v1:172.21.15.177:6816/32265 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa61005ed70 con 0x7fa5fc04cba0 2024-09-13T18:35:18.160 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.157+0000 7fa5f7fff640 1 -- 172.21.15.177:0/229930781 <== mgr.4108 v1:172.21.15.177:6816/32265 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34995 (unknown 2965378022 0 1909064646) 0x7fa61005ed70 con 0x7fa5fc04cba0 2024-09-13T18:35:18.160 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:18.161 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.158+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 >> v1:172.21.15.177:6816/32265 conn(0x7fa5fc04cba0 legacy=0x7fa5fc04f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.158+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 msgr2=0x7fa61008bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.158+0000 7fa616a20640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa61008bd80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa60000ec80 tx=0x7fa6000090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 shutdown_connections 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa61012b4c0 0x7fa61008c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa610091d70 0x7fa61008c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 --2- 172.21.15.177:0/229930781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa610091460 0x7fa61008bd80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 >> 172.21.15.177:0/229930781 conn(0x7fa610099a40 msgr2=0x7fa610097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 shutdown_connections 2024-09-13T18:35:18.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.159+0000 7fa616a20640 1 -- 172.21.15.177:0/229930781 wait complete. 2024-09-13T18:35:18.173 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T18:35:16.613181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191140,"kb_used_data":1692,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296220,"statfs":{"total":386547056640,"available":385327329280,"internally_reserved":0,"allocated":1732608,"data_stored":1323901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.893851"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.735221+0000","last_change":"2024-09-13T18:35:13.735440+0000","last_active":"2024-09-13T18:35:13.735221+0000","last_peered":"2024-09-13T18:35:13.735221+0000","last_clean":"2024-09-13T18:35:13.735221+0000","last_became_active":"2024-09-13T18:35:11.730650+0000","last_became_peered":"2024-09-13T18:35:11.730650+0000","last_unstale":"2024-09-13T18:35:13.735221+0000","last_undegraded":"2024-09-13T18:35:13.735221+0000","last_fullsized":"2024-09-13T18:35:13.735221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:35.164028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734812+0000","last_change":"2024-09-13T18:35:13.735179+0000","last_active":"2024-09-13T18:35:13.734812+0000","last_peered":"2024-09-13T18:35:13.734812+0000","last_clean":"2024-09-13T18:35:13.734812+0000","last_became_active":"2024-09-13T18:35:11.730100+0000","last_became_peered":"2024-09-13T18:35:11.730100+0000","last_unstale":"2024-09-13T18:35:13.734812+0000","last_undegraded":"2024-09-13T18:35:13.734812+0000","last_fullsized":"2024-09-13T18:35:13.734812+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:25.061944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.735530+0000","last_change":"2024-09-13T18:35:13.735829+0000","last_active":"2024-09-13T18:35:13.735530+0000","last_peered":"2024-09-13T18:35:13.735530+0000","last_clean":"2024-09-13T18:35:13.735530+0000","last_became_active":"2024-09-13T18:35:11.730242+0000","last_became_peered":"2024-09-13T18:35:11.730242+0000","last_unstale":"2024-09-13T18:35:13.735530+0000","last_undegraded":"2024-09-13T18:35:13.735530+0000","last_fullsized":"2024-09-13T18:35:13.735530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:01:03.137288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734864+0000","last_change":"2024-09-13T18:35:13.735117+0000","last_active":"2024-09-13T18:35:13.734864+0000","last_peered":"2024-09-13T18:35:13.734864+0000","last_clean":"2024-09-13T18:35:13.734864+0000","last_became_active":"2024-09-13T18:35:11.730015+0000","last_became_peered":"2024-09-13T18:35:11.730015+0000","last_unstale":"2024-09-13T18:35:13.734864+0000","last_undegraded":"2024-09-13T18:35:13.734864+0000","last_fullsized":"2024-09-13T18:35:13.734864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:43.650449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"17'2","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.735587+0000","last_change":"2024-09-13T18:35:13.735587+0000","last_active":"2024-09-13T18:35:13.735587+0000","last_peered":"2024-09-13T18:35:13.735587+0000","last_clean":"2024-09-13T18:35:13.735587+0000","last_became_active":"2024-09-13T18:35:11.733034+0000","last_became_peered":"2024-09-13T18:35:11.733034+0000","last_unstale":"2024-09-13T18:35:13.735587+0000","last_undegraded":"2024-09-13T18:35:13.735587+0000","last_fullsized":"2024-09-13T18:35:13.735587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:01:22.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:14.461023+0000","last_change":"2024-09-13T18:35:14.461158+0000","last_active":"2024-09-13T18:35:14.461023+0000","last_peered":"2024-09-13T18:35:14.461023+0000","last_clean":"2024-09-13T18:35:14.461023+0000","last_became_active":"2024-09-13T18:35:11.729626+0000","last_became_peered":"2024-09-13T18:35:11.729626+0000","last_unstale":"2024-09-13T18:35:14.461023+0000","last_undegraded":"2024-09-13T18:35:14.461023+0000","last_fullsized":"2024-09-13T18:35:14.461023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:10.111028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734705+0000","last_change":"2024-09-13T18:35:13.735139+0000","last_active":"2024-09-13T18:35:13.734705+0000","last_peered":"2024-09-13T18:35:13.734705+0000","last_clean":"2024-09-13T18:35:13.734705+0000","last_became_active":"2024-09-13T18:35:11.733025+0000","last_became_peered":"2024-09-13T18:35:11.733025+0000","last_unstale":"2024-09-13T18:35:13.734705+0000","last_undegraded":"2024-09-13T18:35:13.734705+0000","last_fullsized":"2024-09-13T18:35:13.734705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.647913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"15'1","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T18:35:13.734760+0000","last_change":"2024-09-13T18:35:13.735186+0000","last_active":"2024-09-13T18:35:13.734760+0000","last_peered":"2024-09-13T18:35:13.734760+0000","last_clean":"2024-09-13T18:35:13.734760+0000","last_became_active":"2024-09-13T18:35:11.727196+0000","last_became_peered":"2024-09-13T18:35:11.727196+0000","last_unstale":"2024-09-13T18:35:13.734760+0000","last_undegraded":"2024-09-13T18:35:13.734760+0000","last_fullsized":"2024-09-13T18:35:13.734760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:18.070546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"13'52","reported_seq":80,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T18:35:12.730721+0000","last_change":"2024-09-13T18:35:07.177729+0000","last_active":"2024-09-13T18:35:12.730721+0000","last_peered":"2024-09-13T18:35:12.730721+0000","last_clean":"2024-09-13T18:35:12.730721+0000","last_became_active":"2024-09-13T18:35:07.176786+0000","last_became_peered":"2024-09-13T18:35:07.176786+0000","last_unstale":"2024-09-13T18:35:12.730721+0000","last_undegraded":"2024-09-13T18:35:12.730721+0000","last_fullsized":"2024-09-13T18:35:12.730721+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:02.631221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":143360,"data_stored":37617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":131072,"data_stored":34271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":737280,"data_stored":627985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298064,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073776,"statfs":{"total":96636764160,"available":96331546624,"internally_reserved":0,"allocated":720896,"data_stored":624028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:35:18.174 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:35:18.174 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T18:35:18.174 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T18:35:18.174 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T18:35:18.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.263+0000 7f9a0591d640 1 Processor -- start 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.263+0000 7f9a0591d640 1 -- start start 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f9a0591d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000974d0 0x7f9a00095960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f9a0591d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9a000962f0 0x7f9a0012a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f9a0591d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a00091740 con 0x7f9a00095ea0 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f9a0591d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a00063d40 con 0x7f9a000962f0 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f9a0591d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0006f3f0 con 0x7f9a000974d0 2024-09-13T18:35:18.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99ff7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9a000962f0 0x7f9a0012a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99feffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000974d0 0x7f9a00095960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99fe7fc640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f9a00095ea0 0x7f9a0012a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47524/0 (socket says 172.21.15.177:47524) 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99fe7fc640 1 -- 172.21.15.177:0/4248654863 learned_addr learned my addr 172.21.15.177:0/4248654863 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99feffd640 1 -- 172.21.15.177:0/4248654863 >> v2:172.21.15.177:3300/0 conn(0x7f9a000962f0 msgr2=0x7f9a0012a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99feffd640 1 --2- 172.21.15.177:0/4248654863 >> v2:172.21.15.177:3300/0 conn(0x7f9a000962f0 0x7f9a0012a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99feffd640 1 -- 172.21.15.177:0/4248654863 >> v1:172.21.15.177:6789/0 conn(0x7f9a00095ea0 legacy=0x7f9a0012a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99feffd640 1 -- 172.21.15.177:0/4248654863 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000966c0 con 0x7f9a000974d0 2024-09-13T18:35:18.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99fdffb640 1 -- 172.21.15.177:0/4248654863 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171690451 0 0) 0x7f9a00091740 con 0x7f9a00095ea0 2024-09-13T18:35:18.270 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.264+0000 7f99ff7fe640 1 --2- 172.21.15.177:0/4248654863 >> v2:172.21.15.177:3300/0 conn(0x7f9a000962f0 0x7f9a0012a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f99feffd640 1 --2- 172.21.15.177:0/4248654863 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000974d0 0x7f9a00095960 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009850 tx=0x7f99f00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d63c8acee8ae595 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f99fdffb640 1 -- 172.21.15.177:0/4248654863 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f002ac80 con 0x7f9a000974d0 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f99fdffb640 1 -- 172.21.15.177:0/4248654863 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f002ade0 con 0x7f9a000974d0 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f99fdffb640 1 -- 172.21.15.177:0/4248654863 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0032670 con 0x7f9a000974d0 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 -- 172.21.15.177:0/4248654863 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000974d0 msgr2=0x7f9a00095960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 --2- 172.21.15.177:0/4248654863 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000974d0 0x7f9a00095960 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009850 tx=0x7f99f00090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 -- 172.21.15.177:0/4248654863 shutdown_connections 2024-09-13T18:35:18.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 --2- 172.21.15.177:0/4248654863 >> v2:172.21.15.177:3300/0 conn(0x7f9a000962f0 0x7f9a0012a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 --2- 172.21.15.177:0/4248654863 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000974d0 0x7f9a00095960 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 -- 172.21.15.177:0/4248654863 >> 172.21.15.177:0/4248654863 conn(0x7f9a0009a9b0 msgr2=0x7f9a00096a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 -- 172.21.15.177:0/4248654863 shutdown_connections 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.265+0000 7f9a0591d640 1 -- 172.21.15.177:0/4248654863 wait complete. 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.266+0000 7f9a0591d640 1 Processor -- start 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.266+0000 7f9a0591d640 1 -- start start 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a00095ea0 0x7f9a0011f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 0x7f9a0011f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 0x7f9a001391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a001219f0 con 0x7f9a000974d0 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00120350 con 0x7f9a00095ea0 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00122040 con 0x7f9a000962f0 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 0x7f9a0011f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 0x7f9a001391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99feffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a00095ea0 0x7f9a0011f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 0x7f9a0011f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:58100/0 (socket says 172.21.15.177:58100) 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 0x7f9a001391e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:48192/0 (socket says 172.21.15.177:48192) 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99fe7fc640 1 -- 172.21.15.177:0/1257020739 learned_addr learned my addr 172.21.15.177:0/1257020739 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:18.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 -- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 msgr2=0x7f9a0011f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 0x7f9a0011f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 -- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a00095ea0 msgr2=0x7f9a0011f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a00095ea0 0x7f9a0011f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 -- 172.21.15.177:0/1257020739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8009a20 con 0x7f9a000974d0 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99fe7fc640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 0x7f9a0011f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99feffd640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a00095ea0 0x7f9a0011f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99ff7fe640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 0x7f9a001391e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009f10 tx=0x7f99f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800cdb0 con 0x7f9a000974d0 2024-09-13T18:35:18.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800f040 con 0x7f9a000974d0 2024-09-13T18:35:18.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0009390 con 0x7f9a000974d0 2024-09-13T18:35:18.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.267+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80146d0 con 0x7f9a000974d0 2024-09-13T18:35:18.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.268+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a00120710 con 0x7f9a000974d0 2024-09-13T18:35:18.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.269+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0005ed70 con 0x7f9a000974d0 2024-09-13T18:35:18.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.269+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f99f8003020 con 0x7f9a000974d0 2024-09-13T18:35:18.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.269+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f99f8063bc0 con 0x7f9a000974d0 2024-09-13T18:35:18.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.272+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f99f803ed00 con 0x7f9a000974d0 2024-09-13T18:35:18.425 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.422+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9a00064650 con 0x7f9a000974d0 2024-09-13T18:35:18.426 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.423+0000 7f99e6ffd640 1 -- 172.21.15.177:0/1257020739 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f99f8018020 con 0x7f9a000974d0 2024-09-13T18:35:18.426 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:18.426 INFO:teuthology.orchestra.run.smithi177.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T18:35:18.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 >> v1:172.21.15.177:6816/32265 conn(0x7f99ec04cb70 legacy=0x7f99ec04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 msgr2=0x7f9a001391e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 0x7f9a001391e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009f10 tx=0x7f99f800b040 comp rx=0 tx=0).stop 2024-09-13T18:35:18.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 shutdown_connections 2024-09-13T18:35:18.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a000974d0 0x7f9a001391e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9a000962f0 0x7f9a0011f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 --2- 172.21.15.177:0/1257020739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a00095ea0 0x7f9a0011f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 >> 172.21.15.177:0/1257020739 conn(0x7f9a0009a9b0 msgr2=0x7f9a00098bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 shutdown_connections 2024-09-13T18:35:18.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.424+0000 7f9a0591d640 1 -- 172.21.15.177:0/1257020739 wait complete. 2024-09-13T18:35:18.438 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T18:35:18.438 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T18:35:18.447 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:35:18.448 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:35:18.448 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client quincy' 2024-09-13T18:35:18.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.536+0000 7fac1e894640 1 Processor -- start 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.536+0000 7fac1e894640 1 -- start start 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1e894640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 0x7fac18096ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1e894640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fac18097010 0x7fac18097420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1e894640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac18069e60 con 0x7fac1812a240 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1e894640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1808ee90 con 0x7fac18097010 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1e894640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1806a720 con 0x7fac18098670 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac17fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 0x7fac18096ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1ce31640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fac1812a240 0x7fac1812a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47534/0 (socket says 172.21.15.177:47534) 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac17fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 0x7fac18096ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:47286/0 (socket says 172.21.15.177:47286) 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac0ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fac18097010 0x7fac18097420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac1ce31640 1 -- 172.21.15.177:0/773364938 learned_addr learned my addr 172.21.15.177:0/773364938 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac17fff640 1 -- 172.21.15.177:0/773364938 >> v2:172.21.15.177:3300/0 conn(0x7fac18097010 msgr2=0x7fac18097420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:18.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac17fff640 1 --2- 172.21.15.177:0/773364938 >> v2:172.21.15.177:3300/0 conn(0x7fac18097010 0x7fac18097420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.542 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac17fff640 1 -- 172.21.15.177:0/773364938 >> v1:172.21.15.177:6789/0 conn(0x7fac1812a240 legacy=0x7fac1812a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.542 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac177fe640 1 -- 172.21.15.177:0/773364938 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186638695 0 0) 0x7fac18069e60 con 0x7fac1812a240 2024-09-13T18:35:18.542 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac17fff640 1 -- 172.21.15.177:0/773364938 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac180691a0 con 0x7fac18098670 2024-09-13T18:35:18.542 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.537+0000 7fac0ffff640 1 --2- 172.21.15.177:0/773364938 >> v2:172.21.15.177:3300/0 conn(0x7fac18097010 0x7fac18097420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.538+0000 7fac17fff640 1 --2- 172.21.15.177:0/773364938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 0x7fac18096ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac08009850 tx=0x7fac080090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fe24a3b43959b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.538+0000 7fac177fe640 1 -- 172.21.15.177:0/773364938 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0802ac80 con 0x7fac18098670 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.538+0000 7fac177fe640 1 -- 172.21.15.177:0/773364938 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0802ade0 con 0x7fac18098670 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.538+0000 7fac177fe640 1 -- 172.21.15.177:0/773364938 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08032670 con 0x7fac18098670 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 -- 172.21.15.177:0/773364938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 msgr2=0x7fac18096ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 --2- 172.21.15.177:0/773364938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 0x7fac18096ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac08009850 tx=0x7fac080090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 -- 172.21.15.177:0/773364938 shutdown_connections 2024-09-13T18:35:18.545 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 --2- 172.21.15.177:0/773364938 >> v2:172.21.15.177:3300/0 conn(0x7fac18097010 0x7fac18097420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 --2- 172.21.15.177:0/773364938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18098670 0x7fac18096ad0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 -- 172.21.15.177:0/773364938 >> 172.21.15.177:0/773364938 conn(0x7fac1809bb30 msgr2=0x7fac18097bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 -- 172.21.15.177:0/773364938 shutdown_connections 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.539+0000 7fac1e894640 1 -- 172.21.15.177:0/773364938 wait complete. 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 Processor -- start 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 -- start start 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 0x7fac18139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac18098670 0x7fac18139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fac1812a240 0x7fac1806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18121540 con 0x7fac18098670 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18120bc0 con 0x7fac18097010 2024-09-13T18:35:18.546 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1e894640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac1806f220 con 0x7fac1812a240 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac17fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 0x7fac18139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac0ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac18098670 0x7fac18139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac17fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 0x7fac18139770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:47298/0 (socket says 172.21.15.177:47298) 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac17fff640 1 -- 172.21.15.177:0/1279424968 learned_addr learned my addr 172.21.15.177:0/1279424968 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac1ce31640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fac1812a240 0x7fac1806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac0ffff640 1 -- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fac1812a240 msgr2=0x7fac1806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac0ffff640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fac1812a240 0x7fac1806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac0ffff640 1 -- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 msgr2=0x7fac18139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac0ffff640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 0x7fac18139770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac0ffff640 1 -- 172.21.15.177:0/1279424968 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac00009390 con 0x7fac18098670 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.540+0000 7fac17fff640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 0x7fac18139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.541+0000 7fac0ffff640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac18098670 0x7fac18139cb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac1806aec0 tx=0x7fac000090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.541+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0000c8e0 con 0x7fac18098670 2024-09-13T18:35:18.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.541+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0000ca40 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.541+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac08009390 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.541+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0000ce00 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.541+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1813a1f0 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.542+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fac00013680 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.542+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac18121880 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.542+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fac00063470 con 0x7fac18098670 2024-09-13T18:35:18.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.545+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fac0003e5b0 con 0x7fac18098670 2024-09-13T18:35:18.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.686+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "quincy"} v 0) v1 -- 0x7fac1804bf60 con 0x7fac18098670 2024-09-13T18:35:18.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.755+0000 7fac14ff9640 1 -- 172.21.15.177:0/1279424968 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "quincy"}]=0 set require_min_compat_client to quincy v18) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac00041dc0 con 0x7fac18098670 2024-09-13T18:35:18.759 INFO:teuthology.orchestra.run.smithi177.stderr:set require_min_compat_client to quincy 2024-09-13T18:35:18.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 >> v1:172.21.15.177:6816/32265 conn(0x7fabf804cb70 legacy=0x7fabf804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac18098670 msgr2=0x7fac18139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac18098670 0x7fac18139cb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac1806aec0 tx=0x7fac000090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 shutdown_connections 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fac1812a240 0x7fac1806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac18098670 0x7fac18139cb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 --2- 172.21.15.177:0/1279424968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac18097010 0x7fac18139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 >> 172.21.15.177:0/1279424968 conn(0x7fac1809bb30 msgr2=0x7fac18091280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 shutdown_connections 2024-09-13T18:35:18.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.757+0000 7fac1e894640 1 -- 172.21.15.177:0/1279424968 wait complete. 2024-09-13T18:35:18.770 INFO:teuthology.run_tasks:Running task print... 2024-09-13T18:35:18.778 INFO:teuthology.task.print:**** done ceph 2024-09-13T18:35:18.779 INFO:teuthology.run_tasks:Running task print... 2024-09-13T18:35:18.786 INFO:teuthology.task.print:*** upgrading, no cephfs present 2024-09-13T18:35:18.786 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T18:35:18.792 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:35:18.793 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:35:18.793 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-09-13T18:35:18.885 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 Processor -- start 2024-09-13T18:35:18.885 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 -- start start 2024-09-13T18:35:18.885 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 0x7f90f81247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.885 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 0x7f90f81279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.885 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8127ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f8110130 con 0x7f90f8124cf0 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8110990 con 0x7f90f81250e0 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f91002dd640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8110630 con 0x7f90f806e830 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f90fd878640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 0x7f90f81279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f90fe079640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 0x7f90f81247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f90fd878640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 0x7f90f81279b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:48200/0 (socket says 172.21.15.177:48200) 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f90fd878640 1 -- 172.21.15.177:0/1137005276 learned_addr learned my addr 172.21.15.177:0/1137005276 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f90fe079640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 0x7f90f81247b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:58120/0 (socket says 172.21.15.177:58120) 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.881+0000 7f90fe87a640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8127ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe87a640 1 -- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 msgr2=0x7f90f81247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe87a640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 0x7f90f81247b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe87a640 1 -- 172.21.15.177:0/1137005276 >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 msgr2=0x7f90f81279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:18.886 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe87a640 1 --2- 172.21.15.177:0/1137005276 >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 0x7f90f81279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.887 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe87a640 1 -- 172.21.15.177:0/1137005276 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f810fdf0 con 0x7f90f81250e0 2024-09-13T18:35:18.887 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe079640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 0x7f90f81247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.887 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fd878640 1 --2- 172.21.15.177:0/1137005276 >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 0x7f90f81279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.887 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fe87a640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8127ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009850 tx=0x7f90e80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c490806ac38561a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.882+0000 7f90fd077640 1 -- 172.21.15.177:0/1137005276 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e802abc0 con 0x7f90f81250e0 2024-09-13T18:35:18.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f90fd077640 1 -- 172.21.15.177:0/1137005276 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e802ad20 con 0x7f90f81250e0 2024-09-13T18:35:18.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f90fd077640 1 -- 172.21.15.177:0/1137005276 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8032670 con 0x7f90f81250e0 2024-09-13T18:35:18.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 -- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 msgr2=0x7f90f8127ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8127ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009850 tx=0x7f90e80090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 -- 172.21.15.177:0/1137005276 shutdown_connections 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8127ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 --2- 172.21.15.177:0/1137005276 >> v2:172.21.15.177:3300/0 conn(0x7f90f8124cf0 0x7f90f81279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 --2- 172.21.15.177:0/1137005276 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f806e830 0x7f90f81247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 -- 172.21.15.177:0/1137005276 >> 172.21.15.177:0/1137005276 conn(0x7f90f8099790 msgr2=0x7f90f8099b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 -- 172.21.15.177:0/1137005276 shutdown_connections 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.883+0000 7f91002dd640 1 -- 172.21.15.177:0/1137005276 wait complete. 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 Processor -- start 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 -- start start 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 0x7f90f81110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 0x7f90f8111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:18.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81120e0 con 0x7f90f806e830 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8110c80 con 0x7f90f81250e0 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f91002dd640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81226f0 con 0x7f90f8124cf0 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f90fd878640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 0x7f90f8111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f90fe079640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 0x7f90f81110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f90fd878640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 0x7f90f8111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:58132/0 (socket says 172.21.15.177:58132) 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f90fe079640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 0x7f90f81110d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:48206/0 (socket says 172.21.15.177:48206) 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f90fd878640 1 -- 172.21.15.177:0/758876618 learned_addr learned my addr 172.21.15.177:0/758876618 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.884+0000 7f90fe87a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fe079640 1 -- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 msgr2=0x7f90f8111610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fe079640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 0x7f90f8111610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fe079640 1 -- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 msgr2=0x7f90f8136ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fe079640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8136ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fe079640 1 -- 172.21.15.177:0/758876618 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0009390 con 0x7f90f806e830 2024-09-13T18:35:18.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fd878640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 0x7f90f8111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:18.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90fe079640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 0x7f90f81110d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e970 tx=0x7f90f00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:18.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f00095d0 con 0x7f90f806e830 2024-09-13T18:35:18.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f000c940 con 0x7f90f806e830 2024-09-13T18:35:18.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000cd00 con 0x7f90f806e830 2024-09-13T18:35:18.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ec009ae0 con 0x7f90f806e830 2024-09-13T18:35:18.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.885+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e8009390 con 0x7f90f806e830 2024-09-13T18:35:18.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.886+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f804bf60 con 0x7f90f806e830 2024-09-13T18:35:18.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.886+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f90f0013680 con 0x7f90f806e830 2024-09-13T18:35:18.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.886+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f90f0063460 con 0x7f90f806e830 2024-09-13T18:35:18.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:18.889+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f90f003e5b0 con 0x7f90f806e830 2024-09-13T18:35:19.047 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.043+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f90f811ecc0 con 0x7f90f806e830 2024-09-13T18:35:19.047 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.044+0000 7f90f67fc640 1 -- 172.21.15.177:0/758876618 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7f90f000ce60 con 0x7f90f806e830 2024-09-13T18:35:19.047 INFO:teuthology.orchestra.run.smithi177.stdout:e1 2024-09-13T18:35:19.048 INFO:teuthology.orchestra.run.smithi177.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T18:35:19.048 INFO:teuthology.orchestra.run.smithi177.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:35:19.048 INFO:teuthology.orchestra.run.smithi177.stdout:legacy client fscid: -1 2024-09-13T18:35:19.048 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:19.048 INFO:teuthology.orchestra.run.smithi177.stdout:No filesystems configured 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:dumped fsmap epoch 1 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 >> v1:172.21.15.177:6816/32265 conn(0x7f90e404cb70 legacy=0x7f90e404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 msgr2=0x7f90f81110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 0x7f90f81110d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e970 tx=0x7f90f00090f0 comp rx=0 tx=0).stop 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 shutdown_connections 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90f81250e0 0x7f90f8136ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f90f8124cf0 0x7f90f8111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 --2- 172.21.15.177:0/758876618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f806e830 0x7f90f81110d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:19.049 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 >> 172.21.15.177:0/758876618 conn(0x7f90f8099790 msgr2=0x7f90f812aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:19.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 shutdown_connections 2024-09-13T18:35:19.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:35:19.045+0000 7f91002dd640 1 -- 172.21.15.177:0/758876618 wait complete. 2024-09-13T18:35:19.059 INFO:teuthology.run_tasks:Running task install.upgrade... 2024-09-13T18:35:19.067 INFO:teuthology.task.install:project ceph config {'mon.a': None} overrides {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T18:35:19.068 INFO:teuthology.task.install:remote ubuntu@smithi177.front.sepia.ceph.com config {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T18:35:19.068 INFO:teuthology.task.install:extra packages: [] 2024-09-13T18:35:19.073 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T18:35:19.073 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2024-09-13T18:35:19.074 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2024-09-13T18:35:19.095 INFO:teuthology.orchestra.run.smithi177.stdout:17.2.7-1629.geb16b97b.el9 2024-09-13T18:35:19.095 INFO:teuthology.packaging:The installed version of ceph-common is 17.2.7-1629.geb16b97b.el9 2024-09-13T18:35:19.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:35:19.253 INFO:teuthology.task.install:Ceph rpm upgrade from 17.2.7-1629.geb16b97b.el9 to 18.2.4-860.g6c6b083f 2024-09-13T18:35:19.254 INFO:teuthology.task.install.rpm:Host smithi177 is: centos 9.stream x86_64 2024-09-13T18:35:19.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:35:19.403 INFO:teuthology.task.install.rpm:Repo base URL: https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T18:35:19.404 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/yum.repos.d/ceph.repo 2024-09-13T18:35:19.552 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T18:35:19.552 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-13T18:35:19.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T18:35:19.578 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T18:35:19.643 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T18:35:19.709 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T18:35:19.765 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-09-13T18:35:19.776 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-09-13T18:35:20.007 INFO:teuthology.orchestra.run.smithi177.stdout:60 files removed 2024-09-13T18:35:20.035 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2024-09-13T18:35:20.596 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 290 kB/s | 78 kB 00:00 2024-09-13T18:35:20.871 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 50 kB/s | 11 kB 00:00 2024-09-13T18:35:21.130 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-13T18:35:21.830 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-13T18:35:28.273 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 20 MB 00:05 2024-09-13T18:35:33.524 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-13T18:35:35.158 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-13T18:35:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-13T18:35:41.576 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T18:35:43.837 INFO:teuthology.orchestra.run.smithi177.stderr:Package cephadm available, but not installed. 2024-09-13T18:35:43.837 INFO:teuthology.orchestra.run.smithi177.stdout:No match for argument: cephadm 2024-09-13T18:35:43.843 INFO:teuthology.orchestra.run.smithi177.stderr:Package ceph-mgr-dashboard available, but not installed. 2024-09-13T18:35:43.843 INFO:teuthology.orchestra.run.smithi177.stdout:No match for argument: ceph-mgr-dashboard 2024-09-13T18:35:43.844 INFO:teuthology.orchestra.run.smithi177.stderr:Package ceph-mgr-diskprediction-local available, but not installed. 2024-09-13T18:35:43.844 INFO:teuthology.orchestra.run.smithi177.stdout:No match for argument: ceph-mgr-diskprediction-local 2024-09-13T18:35:43.844 INFO:teuthology.orchestra.run.smithi177.stderr:Package ceph-mgr-rook available, but not installed. 2024-09-13T18:35:43.845 INFO:teuthology.orchestra.run.smithi177.stdout:No match for argument: ceph-mgr-rook 2024-09-13T18:35:43.845 INFO:teuthology.orchestra.run.smithi177.stderr:Package ceph-mgr-cephadm available, but not installed. 2024-09-13T18:35:43.845 INFO:teuthology.orchestra.run.smithi177.stdout:No match for argument: ceph-mgr-cephadm 2024-09-13T18:35:43.847 INFO:teuthology.orchestra.run.smithi177.stderr:Package ceph-volume available, but not installed. 2024-09-13T18:35:43.847 INFO:teuthology.orchestra.run.smithi177.stdout:No match for argument: ceph-volume 2024-09-13T18:35:43.934 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-13T18:35:43.937 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T18:35:43.938 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T18:35:43.939 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T18:35:43.940 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-13T18:35:43.941 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-09-13T18:35:43.942 INFO:teuthology.orchestra.run.smithi177.stdout:Install 6 Packages 2024-09-13T18:35:43.942 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 30 Packages 2024-09-13T18:35:43.942 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:35:43.942 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 119 M 2024-09-13T18:35:43.942 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-13T18:35:44.319 INFO:teuthology.orchestra.run.smithi177.stdout:(1/36): fmt-8.1.1-5.el9.x86_64.rpm 1.6 MB/s | 111 kB 00:00 2024-09-13T18:35:44.344 INFO:teuthology.orchestra.run.smithi177.stdout:(2/36): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T18:35:44.378 INFO:teuthology.orchestra.run.smithi177.stdout:(3/36): boost-program-options-1.75.0-8.el9.x86_ 851 kB/s | 107 kB 00:00 2024-09-13T18:35:44.429 INFO:teuthology.orchestra.run.smithi177.stdout:(4/36): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-13T18:35:44.470 INFO:teuthology.orchestra.run.smithi177.stdout:(5/36): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-13T18:35:44.554 INFO:teuthology.orchestra.run.smithi177.stdout:(6/36): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-13T18:35:44.621 INFO:teuthology.orchestra.run.smithi177.stdout:(7/36): ceph-18.2.4-860.g6c6b083f.el9.x86_64.rp 34 kB/s | 6.5 kB 00:00 2024-09-13T18:35:44.805 INFO:teuthology.orchestra.run.smithi177.stdout:(8/36): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86_ 4.6 MB/s | 852 kB 00:00 2024-09-13T18:35:44.880 INFO:teuthology.orchestra.run.smithi177.stdout:(9/36): ceph-immutable-object-cache-18.2.4-860. 1.9 MB/s | 142 kB 00:00 2024-09-13T18:35:44.947 INFO:teuthology.orchestra.run.smithi177.stdout:(10/36): ceph-base-18.2.4-860.g6c6b083f.el9.x86 11 MB/s | 5.1 MB 00:00 2024-09-13T18:35:45.039 INFO:teuthology.orchestra.run.smithi177.stdout:(11/36): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 16 MB/s | 1.5 MB 00:00 2024-09-13T18:35:45.089 INFO:teuthology.orchestra.run.smithi177.stdout:(12/36): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 10 MB/s | 2.1 MB 00:00 2024-09-13T18:35:45.256 INFO:teuthology.orchestra.run.smithi177.stdout:(13/36): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 22 MB/s | 4.7 MB 00:00 2024-09-13T18:35:45.657 INFO:teuthology.orchestra.run.smithi177.stdout:(14/36): ceph-radosgw-18.2.4-860.g6c6b083f.el9. 19 MB/s | 7.7 MB 00:00 2024-09-13T18:35:45.723 INFO:teuthology.orchestra.run.smithi177.stdout:(15/36): ceph-selinux-18.2.4-860.g6c6b083f.el9. 379 kB/s | 25 kB 00:00 2024-09-13T18:35:46.065 INFO:teuthology.orchestra.run.smithi177.stdout:(16/36): ceph-common-18.2.4-860.g6c6b083f.el9.x 12 MB/s | 18 MB 00:01 2024-09-13T18:35:46.257 INFO:teuthology.orchestra.run.smithi177.stdout:(17/36): ceph-osd-18.2.4-860.g6c6b083f.el9.x86_ 14 MB/s | 17 MB 00:01 2024-09-13T18:35:46.283 INFO:teuthology.orchestra.run.smithi177.stdout:(18/36): libcephfs-devel-18.2.4-860.g6c6b083f.e 143 kB/s | 31 kB 00:00 2024-09-13T18:35:46.324 INFO:teuthology.orchestra.run.smithi177.stdout:(19/36): libcephfs2-18.2.4-860.g6c6b083f.el9.x8 10 MB/s | 712 kB 00:00 2024-09-13T18:35:46.350 INFO:teuthology.orchestra.run.smithi177.stdout:(20/36): libcephsqlite-18.2.4-860.g6c6b083f.el9 2.4 MB/s | 166 kB 00:00 2024-09-13T18:35:46.375 INFO:teuthology.orchestra.run.smithi177.stdout:(21/36): librados-devel-18.2.4-860.g6c6b083f.el 2.5 MB/s | 127 kB 00:00 2024-09-13T18:35:46.434 INFO:teuthology.orchestra.run.smithi177.stdout:(22/36): libradosstriper1-18.2.4-860.g6c6b083f. 8.0 MB/s | 478 kB 00:00 2024-09-13T18:35:46.576 INFO:teuthology.orchestra.run.smithi177.stdout:(23/36): librados2-18.2.4-860.g6c6b083f.el9.x86 15 MB/s | 3.3 MB 00:00 2024-09-13T18:35:46.701 INFO:teuthology.orchestra.run.smithi177.stdout:(24/36): librbd1-18.2.4-860.g6c6b083f.el9.x86_6 11 MB/s | 3.0 MB 00:00 2024-09-13T18:35:46.759 INFO:teuthology.orchestra.run.smithi177.stdout:(25/36): python3-ceph-argparse-18.2.4-860.g6c6b 774 kB/s | 45 kB 00:00 2024-09-13T18:35:46.827 INFO:teuthology.orchestra.run.smithi177.stdout:(26/36): python3-ceph-common-18.2.4-860.g6c6b08 1.9 MB/s | 129 kB 00:00 2024-09-13T18:35:46.902 INFO:teuthology.orchestra.run.smithi177.stdout:(27/36): librgw2-18.2.4-860.g6c6b083f.el9.x86_6 14 MB/s | 4.5 MB 00:00 2024-09-13T18:35:46.927 INFO:teuthology.orchestra.run.smithi177.stdout:(28/36): python3-cephfs-18.2.4-860.g6c6b083f.el 1.6 MB/s | 162 kB 00:00 2024-09-13T18:35:46.961 INFO:teuthology.orchestra.run.smithi177.stdout:(29/36): python3-rados-18.2.4-860.g6c6b083f.el9 5.4 MB/s | 321 kB 00:00 2024-09-13T18:35:46.986 INFO:teuthology.orchestra.run.smithi177.stdout:(30/36): python3-rbd-18.2.4-860.g6c6b083f.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-13T18:35:47.011 INFO:teuthology.orchestra.run.smithi177.stdout:(31/36): python3-rgw-18.2.4-860.g6c6b083f.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-13T18:35:47.036 INFO:teuthology.orchestra.run.smithi177.stdout:(32/36): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86_ 1.7 MB/s | 87 kB 00:00 2024-09-13T18:35:47.087 INFO:teuthology.orchestra.run.smithi177.stdout:(33/36): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_6 3.4 MB/s | 172 kB 00:00 2024-09-13T18:35:47.162 INFO:teuthology.orchestra.run.smithi177.stdout:(34/36): ceph-mgr-modules-core-18.2.4-860.g6c6b 3.2 MB/s | 246 kB 00:00 2024-09-13T18:35:47.212 INFO:teuthology.orchestra.run.smithi177.stdout:(35/36): rbd-mirror-18.2.4-860.g6c6b083f.el9.x8 15 MB/s | 3.0 MB 00:00 2024-09-13T18:35:47.588 INFO:teuthology.orchestra.run.smithi177.stdout:(36/36): ceph-test-18.2.4-860.g6c6b083f.el9.x86 21 MB/s | 40 MB 00:01 2024-09-13T18:35:47.594 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:35:47.594 INFO:teuthology.orchestra.run.smithi177.stdout:Total 33 MB/s | 119 MB 00:03 2024-09-13T18:35:47.608 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-13T18:35:47.666 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-13T18:35:47.666 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-13T18:35:48.517 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-13T18:35:48.517 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-13T18:35:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-13T18:35:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 1/66 2024-09-13T18:35:51.640 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 1/66 2024-09-13T18:35:51.715 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2/66 2024-09-13T18:35:51.826 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2/66 2024-09-13T18:35:51.874 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 3/66 2024-09-13T18:35:51.948 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 3/66 2024-09-13T18:35:52.006 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 4/66 2024-09-13T18:35:52.034 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 5/66 2024-09-13T18:35:52.135 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 5/66 2024-09-13T18:35:52.168 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 6/66 2024-09-13T18:35:52.377 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 6/66 2024-09-13T18:35:52.429 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x 7/66 2024-09-13T18:35:52.507 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 8/66 2024-09-13T18:35:52.559 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/66 2024-09-13T18:35:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 10/66 2024-09-13T18:35:52.828 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.n 11/66 2024-09-13T18:35:52.879 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86 12/66 2024-09-13T18:35:52.985 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/66 2024-09-13T18:35:53.013 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 14/66 2024-09-13T18:35:53.038 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 15/66 2024-09-13T18:35:53.071 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fmt-8.1.1-5.el9.x86_64 16/66 2024-09-13T18:35:53.115 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 17/66 2024-09-13T18:35:53.607 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 18/66 2024-09-13T18:35:53.677 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/66 2024-09-13T18:35:53.758 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/66 2024-09-13T18:35:53.796 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 20/66 2024-09-13T18:35:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 21/66 2024-09-13T18:35:56.106 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 21/66 2024-09-13T18:35:56.745 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 21/66 2024-09-13T18:35:56.868 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 22/66 2024-09-13T18:35:56.932 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 22/66 2024-09-13T18:35:56.978 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 23/66 2024-09-13T18:36:05.750 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 23/66 2024-09-13T18:36:05.841 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 24/66 2024-09-13T18:36:05.999 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 24/66 2024-09-13T18:36:06.050 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 25/66 2024-09-13T18:36:06.597 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 25/66 2024-09-13T18:36:06.643 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 26/66 2024-09-13T18:36:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 26/66 2024-09-13T18:36:08.496 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 27/66 2024-09-13T18:36:08.508 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 27/66 2024-09-13T18:36:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 28/66 2024-09-13T18:36:09.464 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 29/66 2024-09-13T18:36:09.486 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 29/66 2024-09-13T18:36:09.539 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f 30/66 2024-09-13T18:36:09.880 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f 30/66 2024-09-13T18:36:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 31/66 2024-09-13T18:36:14.413 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 31/66 2024-09-13T18:36:14.800 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 32/66 2024-09-13T18:36:14.845 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 33/66 2024-09-13T18:36:14.911 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 34/66 2024-09-13T18:36:15.020 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 35/66 2024-09-13T18:36:15.080 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 36/66 2024-09-13T18:36:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-2:17.2.7-1629.geb16b97b.el9.x86_64 37/66 2024-09-13T18:36:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-test-2:17.2.7-1629.geb16b97b.el9.x86_64 38/66 2024-09-13T18:36:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 39/66 2024-09-13T18:36:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 39/66 2024-09-13T18:36:15.236 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 39/66 2024-09-13T18:36:15.260 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : rbd-nbd-2:17.2.7-1629.geb16b97b.el9.x86_64 40/66 2024-09-13T18:36:15.266 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 41/66 2024-09-13T18:36:15.266 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 41/66 2024-09-13T18:36:15.294 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 41/66 2024-09-13T18:36:15.299 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 42/66 2024-09-13T18:36:15.299 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 42/66 2024-09-13T18:36:15.328 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 42/66 2024-09-13T18:36:15.353 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : rbd-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 43/66 2024-09-13T18:36:15.358 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1629.geb16b97 44/66 2024-09-13T18:36:15.358 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-immutable-object-cache-2:17.2.7-1629.geb16b97 44/66 2024-09-13T18:36:15.387 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1629.geb16b97 44/66 2024-09-13T18:36:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libcephfs-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 45/66 2024-09-13T18:36:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 46/66 2024-09-13T18:36:15.496 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 46/66 2024-09-13T18:36:15.521 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 47/66 2024-09-13T18:36:15.526 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 48/66 2024-09-13T18:36:15.526 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 48/66 2024-09-13T18:36:15.555 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 48/66 2024-09-13T18:36:15.560 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 49/66 2024-09-13T18:36:15.560 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 49/66 2024-09-13T18:36:15.589 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 49/66 2024-09-13T18:36:15.594 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 50/66 2024-09-13T18:36:15.594 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 50/66 2024-09-13T18:36:15.631 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 50/66 2024-09-13T18:36:15.631 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 51/66 2024-09-13T18:36:15.665 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 51/66 2024-09-13T18:36:15.671 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 52/66 2024-09-13T18:36:15.671 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 52/66 2024-09-13T18:36:15.768 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 52/66 2024-09-13T18:36:15.768 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 53/66 2024-09-13T18:36:15.810 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 53/66 2024-09-13T18:36:15.811 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_6 54/66 2024-09-13T18:36:15.910 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_6 54/66 2024-09-13T18:36:15.936 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : python3-cephfs-2:17.2.7-1629.geb16b97b.el9.x86_64 55/66 2024-09-13T18:36:15.965 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : python3-rbd-2:17.2.7-1629.geb16b97b.el9.x86_64 56/66 2024-09-13T18:36:15.965 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 57/66 2024-09-13T18:36:16.046 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 57/66 2024-09-13T18:36:16.071 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : python3-rgw-2:17.2.7-1629.geb16b97b.el9.x86_64 58/66 2024-09-13T18:36:16.072 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 59/66 2024-09-13T18:36:16.156 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 59/66 2024-09-13T18:36:16.181 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : python3-rados-2:17.2.7-1629.geb16b97b.el9.x86_64 60/66 2024-09-13T18:36:16.181 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 61/66 2024-09-13T18:36:16.265 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 61/66 2024-09-13T18:36:16.290 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 62/66 2024-09-13T18:36:16.315 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : python3-ceph-argparse-2:17.2.7-1629.geb16b97b.el9. 63/66 2024-09-13T18:36:16.341 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : python3-ceph-common-2:17.2.7-1629.geb16b97b.el9.x8 64/66 2024-09-13T18:36:16.366 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : ceph-mgr-modules-core-2:17.2.7-1629.geb16b97b.el9. 65/66 2024-09-13T18:36:16.367 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 66/66 2024-09-13T18:36:17.519 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 66/66 2024-09-13T18:36:17.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 1/66 2024-09-13T18:36:17.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 2/66 2024-09-13T18:36:17.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 3/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 4/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 5/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-20.el9.x86_64 6/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 7/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:17.2.7-1629.geb16b97b.el9.x86_64 8/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 9/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:17.2.7-1629.geb16b97b.el9.x86_64 10/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 11/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:17.2.7-1629.geb16b97b.el9.x86_64 12/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 13/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 14/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f 15/66 2024-09-13T18:36:17.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1629.geb16b97 16/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 17/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:17.2.7-1629.geb16b97b.el9.x86_64 18/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 19/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:17.2.7-1629.geb16b97b.el9.x86_64 20/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 21/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:17.2.7-1629.geb16b97b.el9.x86_64 22/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 23/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:17.2.7-1629.geb16b97b.el9.x86_64 24/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:17.2.7-1629.geb16b97b.el9.x86_64 26/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 27/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:17.2.7-1629.geb16b97b.el9.x86_64 28/66 2024-09-13T18:36:17.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 29/66 2024-09-13T18:36:17.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:17.2.7-1629.geb16b97b.el9.x86_64 30/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 31/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 32/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 33/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:17.2.7-1629.geb16b97b.el9.x86_64 34/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 35/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:17.2.7-1629.geb16b97b.el9.x86_64 36/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 37/66 2024-09-13T18:36:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:17.2.7-1629.geb16b97b.el9.x86_64 38/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 39/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:17.2.7-1629.geb16b97b.el9.x86_64 40/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 41/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:17.2.7-1629.geb16b97b.el9.x86_6 42/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 43/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:17.2.7-1629.geb16b97b.el9.x86_64 44/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 45/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:17.2.7-1629.geb16b97b.el9.x86_64 46/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x 47/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1629.geb16b97b.el9. 48/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86 49/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:17.2.7-1629.geb16b97b.el9.x8 50/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 51/66 2024-09-13T18:36:17.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:17.2.7-1629.geb16b97b.el9.x86_64 52/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 53/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:17.2.7-1629.geb16b97b.el9.x86_64 54/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 55/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:17.2.7-1629.geb16b97b.el9.x86_64 56/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 57/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:17.2.7-1629.geb16b97b.el9.x86_64 58/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 59/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:17.2.7-1629.geb16b97b.el9.x86_64 60/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 61/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:17.2.7-1629.geb16b97b.el9.x86_64 62/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 63/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:17.2.7-1629.geb16b97b.el9.x86_64 64/66 2024-09-13T18:36:17.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.n 65/66 2024-09-13T18:36:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1629.geb16b97b.el9. 66/66 2024-09-13T18:36:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:18.802 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-09-13T18:36:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.803 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.804 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 fmt-8.1.1-5.el9.x86_64 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-11.el9.x86_64 libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-11.el9.x86_64 re2-1:20211101-20.el9.x86_64 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:18.805 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-13T18:36:19.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:36:19.403 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T18:36:19.454 INFO:teuthology.orchestra.run.smithi177.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T18:36:19.454 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T18:36:19.454 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T18:36:19.457 INFO:teuthology.run_tasks:Running task print... 2024-09-13T18:36:19.521 INFO:teuthology.task.print:**** done install.upgrade 2024-09-13T18:36:19.521 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-09-13T18:36:19.763 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:36:19.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:19.775 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:36:19.776 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:36:19.776 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:36:19.819 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:36:19.819 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:36:19.819 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:19.832 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:36:19.832 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:36:19.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:36:19.874 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:36:19.875 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:36:19.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:19.887 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:36:19.887 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:36:19.887 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:36:19.931 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:36:19.931 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T18:36:19.932 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:19.952 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T18:36:19.953 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T18:36:19.953 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T18:36:19.995 INFO:tasks.ceph.mgr.x:Started 2024-09-13T18:36:19.996 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T18:36:19.996 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:20.018 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T18:36:20.019 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T18:36:20.019 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T18:36:20.063 INFO:tasks.ceph.mgr.y:Started 2024-09-13T18:36:20.063 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-13T18:36:20.073 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T18:36:20.073 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T18:36:20.073 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T18:36:20.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.180+0000 7f562b1a0640 1 Processor -- start 2024-09-13T18:36:20.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.180+0000 7f562b1a0640 1 -- start start 2024-09-13T18:36:20.183 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562b1a0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.183 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562b1a0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562412d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.183 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562b1a0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f562412d920 0x7f562412ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562b1a0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562406c7d0 con 0x7f562412d920 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562b1a0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562406cf60 con 0x7f562406bd60 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562b1a0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f562412e310 con 0x7f562406b440 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562999d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562412d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562999d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562412d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:34956/0 (socket says 172.21.15.177:34956) 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562999d640 1 -- 172.21.15.177:0/778795742 learned_addr learned my addr 172.21.15.177:0/778795742 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a19e640 1 --2- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a99f640 1 --2- 172.21.15.177:0/778795742 >> v2:172.21.15.177:3300/0 conn(0x7f562412d920 0x7f562412ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a19e640 1 -- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 msgr2=0x7f562412d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a19e640 1 --2- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562412d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a19e640 1 -- 172.21.15.177:0/778795742 >> v2:172.21.15.177:3300/0 conn(0x7f562412d920 msgr2=0x7f562412ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:20.185 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a19e640 1 --2- 172.21.15.177:0/778795742 >> v2:172.21.15.177:3300/0 conn(0x7f562412d920 0x7f562412ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.185 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.181+0000 7f562a19e640 1 -- 172.21.15.177:0/778795742 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624070ee0 con 0x7f562406b440 2024-09-13T18:36:20.187 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.182+0000 7f562a19e640 1 --2- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562406b820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5620013de0 tx=0x7f5620032700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388eb850e19377ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.182+0000 7f562919c640 1 -- 172.21.15.177:0/778795742 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620037710 con 0x7f562406b440 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.182+0000 7f562919c640 1 -- 172.21.15.177:0/778795742 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620035050 con 0x7f562406b440 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.182+0000 7f562919c640 1 -- 172.21.15.177:0/778795742 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620037a90 con 0x7f562406b440 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 -- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 msgr2=0x7f562406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 --2- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562406b820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5620013de0 tx=0x7f5620032700 comp rx=0 tx=0).stop 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 -- 172.21.15.177:0/778795742 shutdown_connections 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 --2- 172.21.15.177:0/778795742 >> v2:172.21.15.177:3300/0 conn(0x7f562412d920 0x7f562412ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 --2- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562412d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 --2- 172.21.15.177:0/778795742 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562406b820 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 -- 172.21.15.177:0/778795742 >> 172.21.15.177:0/778795742 conn(0x7f562409a630 msgr2=0x7f562409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:20.188 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 -- 172.21.15.177:0/778795742 shutdown_connections 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.183+0000 7f562b1a0640 1 -- 172.21.15.177:0/778795742 wait complete. 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.184+0000 7f562b1a0640 1 Processor -- start 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.184+0000 7f562b1a0640 1 -- start start 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.184+0000 7f562b1a0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562408b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562b1a0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562408b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562b1a0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f562412d920 0x7f562408bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562b1a0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562408d030 con 0x7f562412d920 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562b1a0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5624070680 con 0x7f562406bd60 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562b1a0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5624127cb0 con 0x7f562406b440 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562408b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562408b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60872/0 (socket says 172.21.15.177:60872) 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 -- 172.21.15.177:0/52394389 learned_addr learned my addr 172.21.15.177:0/52394389 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:20.189 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a99f640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f562412d920 0x7f562408bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562999d640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562408b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 -- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 msgr2=0x7f562408b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562408b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 -- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f562412d920 msgr2=0x7f562408bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f562412d920 0x7f562408bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 -- 172.21.15.177:0/52394389 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620013a10 con 0x7f562406b440 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562a19e640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562408b440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5620013f00 tx=0x7f56200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620043620 con 0x7f562406b440 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620043780 con 0x7f562406b440 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.185+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562408d450 con 0x7f562406b440 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.186+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620043b30 con 0x7f562406b440 2024-09-13T18:36:20.190 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.186+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562408db90 con 0x7f562406b440 2024-09-13T18:36:20.191 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.186+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562408e470 con 0x7f562406b440 2024-09-13T18:36:20.191 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.187+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f5620040070 con 0x7f562406b440 2024-09-13T18:36:20.191 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.188+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f562008be40 con 0x7f562406b440 2024-09-13T18:36:20.191 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.188+0000 7f562999d640 1 -- 172.21.15.177:0/52394389 >> v1:172.21.15.177:6816/32265 conn(0x7f560804cfc0 legacy=0x7f560804f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:20.197 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.195+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f562006ab90 con 0x7f562406b440 2024-09-13T18:36:20.239 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.237+0000 7f516140c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:36:20.240 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.238+0000 7eff70123040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:36:20.281 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.279+0000 7f516140c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:36:20.282 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.280+0000 7eff70123040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:36:20.333 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.331+0000 7f516140c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:36:20.333 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.332+0000 7eff70123040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:36:20.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.360+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5624067400 con 0x7f562406b440 2024-09-13T18:36:20.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.362+0000 7f561affd640 1 -- 172.21.15.177:0/52394389 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v6) v1 ==== 74+0+116316 (secure 0 0 0) 0x7f562003e020 con 0x7f562406b440 2024-09-13T18:36:20.366 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:20.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 >> v1:172.21.15.177:6816/32265 conn(0x7f560804cfc0 legacy=0x7f560804f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:20.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 msgr2=0x7f562408b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562408b440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5620013f00 tx=0x7f56200046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 shutdown_connections 2024-09-13T18:36:20.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f562412d920 0x7f562408bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f562406bd60 0x7f562408b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 --2- 172.21.15.177:0/52394389 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f562406b440 0x7f562408b440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 >> 172.21.15.177:0/52394389 conn(0x7f562409a630 msgr2=0x7f562409e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:20.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 shutdown_connections 2024-09-13T18:36:20.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.365+0000 7f562b1a0640 1 -- 172.21.15.177:0/52394389 wait complete. 2024-09-13T18:36:20.378 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":6,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6816","nonce":32265}]},"active_addr":"172.21.15.177:6816/32265","active_change":"2024-09-13T18:34:56.604440+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3625374812}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2735606740}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":1782828617}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2408340411}]}]} 2024-09-13T18:36:20.382 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T18:36:20.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T18:36:20.382 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:20.459 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.457+0000 7f516140c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:36:20.460 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.458+0000 7eff70123040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:36:20.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.480+0000 7f56383db640 1 Processor -- start 2024-09-13T18:36:20.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.480+0000 7f56383db640 1 -- start start 2024-09-13T18:36:20.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f56383db640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630132d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.488 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f56383db640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f5630135630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.488 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f56383db640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5630098be0 con 0x7f563012a370 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f56383db640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563009a310 con 0x7f5630133240 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f56383db640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f563012ae60 con 0x7f5630130910 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5636978640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f5630135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630132d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630132d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60884/0 (socket says 172.21.15.177:60884) 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5636978640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f5630135630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:34970/0 (socket says 172.21.15.177:34970) 2024-09-13T18:36:20.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5636177640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f563012a370 0x7f563012a750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58062/0 (socket says 172.21.15.177:58062) 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 -- 172.21.15.177:0/1410983909 learned_addr learned my addr 172.21.15.177:0/1410983909 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 -- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 msgr2=0x7f5630135630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 --2- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f5630135630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 -- 172.21.15.177:0/1410983909 >> v1:172.21.15.177:6789/0 conn(0x7f563012a370 legacy=0x7f563012a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635175640 1 -- 172.21.15.177:0/1410983909 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964869171 0 0) 0x7f5630098be0 con 0x7f563012a370 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.481+0000 7f5635976640 1 -- 172.21.15.177:0/1410983909 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5630099400 con 0x7f5630130910 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.482+0000 7f5636978640 1 --2- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f5630135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:20.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.482+0000 7f5635976640 1 --2- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630132d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56200098a0 tx=0x7f562002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e76bdb3b1666358 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:20.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.482+0000 7f5635175640 1 -- 172.21.15.177:0/1410983909 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562002ae60 con 0x7f5630130910 2024-09-13T18:36:20.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.482+0000 7f5635175640 1 -- 172.21.15.177:0/1410983909 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620032440 con 0x7f5630130910 2024-09-13T18:36:20.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.482+0000 7f5635175640 1 -- 172.21.15.177:0/1410983909 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200327d0 con 0x7f5630130910 2024-09-13T18:36:20.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.483+0000 7f56383db640 1 -- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 msgr2=0x7f5630132d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.483+0000 7f56383db640 1 --2- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630132d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56200098a0 tx=0x7f562002a110 comp rx=0 tx=0).stop 2024-09-13T18:36:20.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.483+0000 7f56383db640 1 -- 172.21.15.177:0/1410983909 shutdown_connections 2024-09-13T18:36:20.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.484+0000 7f56383db640 1 --2- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f5630135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.484+0000 7f56383db640 1 --2- 172.21.15.177:0/1410983909 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630132d00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.484+0000 7f56383db640 1 -- 172.21.15.177:0/1410983909 >> 172.21.15.177:0/1410983909 conn(0x7f56300a0160 msgr2=0x7f56300a0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:20.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.484+0000 7f56383db640 1 -- 172.21.15.177:0/1410983909 shutdown_connections 2024-09-13T18:36:20.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.484+0000 7f56383db640 1 -- 172.21.15.177:0/1410983909 wait complete. 2024-09-13T18:36:20.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 Processor -- start 2024-09-13T18:36:20.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 -- start start 2024-09-13T18:36:20.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f563012a370 0x7f5630125730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630125c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f56301261b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630127370 con 0x7f563012a370 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5630128ff0 con 0x7f5630133240 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.485+0000 7f56383db640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5630128960 con 0x7f5630130910 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630125c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5636177640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f563012a370 0x7f5630125730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630125c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60898/0 (socket says 172.21.15.177:60898) 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 -- 172.21.15.177:0/1322633508 learned_addr learned my addr 172.21.15.177:0/1322633508 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:20.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5636978640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f56301261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 -- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 msgr2=0x7f56301261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f56301261b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 -- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f563012a370 msgr2=0x7f5630125730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f563012a370 0x7f5630125730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 -- 172.21.15.177:0/1322633508 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5628009a70 con 0x7f5630130910 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5636177640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f563012a370 0x7f5630125730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5635976640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630125c70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5620000c00 tx=0x7f56200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620004230 con 0x7f5630130910 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620004390 con 0x7f5630130910 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620003810 con 0x7f5630130910 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56200094d0 con 0x7f5630130910 2024-09-13T18:36:20.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.486+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5630127f10 con 0x7f5630130910 2024-09-13T18:36:20.500 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.487+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f5620039050 con 0x7f5630130910 2024-09-13T18:36:20.500 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.488+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5630128410 con 0x7f5630130910 2024-09-13T18:36:20.500 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.488+0000 7f5636177640 1 -- 172.21.15.177:0/1322633508 >> v1:172.21.15.177:6816/32265 conn(0x7f561404cff0 legacy=0x7f561404f4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:20.500 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.488+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f5620083870 con 0x7f5630130910 2024-09-13T18:36:20.500 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.491+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f5620062460 con 0x7f5630130910 2024-09-13T18:36:20.579 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.577+0000 7eff70123040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:36:20.585 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.583+0000 7f516140c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:36:20.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.662+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f563012a750 con 0x7f5630130910 2024-09-13T18:36:20.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.663+0000 7f5626ffd640 1 -- 172.21.15.177:0/1322633508 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+8628 (secure 0 0 0) 0x7f5620037070 con 0x7f5630130910 2024-09-13T18:36:20.665 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:20.665 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":19,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:35:18.982537+0000","last_up_change":"2024-09-13T18:35:03.616896+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":32756}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":32756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":32756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":32756}]},"public_addr":"172.21.15.177:6804/32756","cluster_addr":"172.21.15.177:6806/32756","heartbeat_back_addr":"172.21.15.177:6810/32756","heartbeat_front_addr":"172.21.15.177:6809/32756","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.664+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 >> v1:172.21.15.177:6816/32265 conn(0x7f561404cff0 legacy=0x7f561404f4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.664+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 msgr2=0x7f5630125c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630125c70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5620000c00 tx=0x7f56200046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 shutdown_connections 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630133240 0x7f56301261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5630130910 0x7f5630125c70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 --2- 172.21.15.177:0/1322633508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f563012a370 0x7f5630125730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 >> 172.21.15.177:0/1322633508 conn(0x7f56300a0160 msgr2=0x7f563012ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 shutdown_connections 2024-09-13T18:36:20.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.665+0000 7f56383db640 1 -- 172.21.15.177:0/1322633508 wait complete. 2024-09-13T18:36:20.678 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T18:36:20.678 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:20.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.774+0000 7f8b6cb45640 1 Processor -- start 2024-09-13T18:36:20.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.774+0000 7f8b6cb45640 1 -- start start 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6cb45640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b6806bf30 0x7f8b68132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6cb45640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b68132d40 0x7f8b68135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6cb45640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6806d840 con 0x7f8b6806b440 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6cb45640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6806c4a0 con 0x7f8b68132d40 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6cb45640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6806db60 con 0x7f8b6806bf30 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6659c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f8b6806b440 0x7f8b6806b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58064/0 (socket says 172.21.15.177:58064) 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b66d9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b68132d40 0x7f8b68135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6659c640 1 -- 172.21.15.177:0/2018369707 learned_addr learned my addr 172.21.15.177:0/2018369707 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b65d9b640 1 --2- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b6806bf30 0x7f8b68132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b6559a640 1 -- 172.21.15.177:0/2018369707 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769383895 0 0) 0x7f8b6806d840 con 0x7f8b6806b440 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b66d9d640 1 -- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b6806bf30 msgr2=0x7f8b68132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:20.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b66d9d640 1 --2- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b6806bf30 0x7f8b68132800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b66d9d640 1 -- 172.21.15.177:0/2018369707 >> v1:172.21.15.177:6789/0 conn(0x7f8b6806b440 legacy=0x7f8b6806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.775+0000 7f8b66d9d640 1 -- 172.21.15.177:0/2018369707 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6806cfb0 con 0x7f8b68132d40 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.776+0000 7f8b66d9d640 1 --2- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b68132d40 0x7f8b68135130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009dc0 tx=0x7f8b6002a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab215378855b870f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.776+0000 7f8b6559a640 1 -- 172.21.15.177:0/2018369707 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60007740 con 0x7f8b68132d40 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.776+0000 7f8b6559a640 1 -- 172.21.15.177:0/2018369707 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6002d050 con 0x7f8b68132d40 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.776+0000 7f8b6559a640 1 -- 172.21.15.177:0/2018369707 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60007ac0 con 0x7f8b68132d40 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 -- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b68132d40 msgr2=0x7f8b68135130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b68132d40 0x7f8b68135130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009dc0 tx=0x7f8b6002a510 comp rx=0 tx=0).stop 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 -- 172.21.15.177:0/2018369707 shutdown_connections 2024-09-13T18:36:20.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b68132d40 0x7f8b68135130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/2018369707 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b6806bf30 0x7f8b68132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 -- 172.21.15.177:0/2018369707 >> 172.21.15.177:0/2018369707 conn(0x7f8b6809fe10 msgr2=0x7f8b680a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:20.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 -- 172.21.15.177:0/2018369707 shutdown_connections 2024-09-13T18:36:20.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.777+0000 7f8b6cb45640 1 -- 172.21.15.177:0/2018369707 wait complete. 2024-09-13T18:36:20.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.778+0000 7f8b6cb45640 1 Processor -- start 2024-09-13T18:36:20.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.778+0000 7f8b6cb45640 1 -- start start 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.778+0000 7f8b6cb45640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 0x7f8b6806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.778+0000 7f8b6cb45640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6806bf30 0x7f8b6806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.778+0000 7f8b6cb45640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b68132d40 0x7f8b6806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6cb45640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68127f00 con 0x7f8b6806b440 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6cb45640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68070a60 con 0x7f8b6806bf30 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6cb45640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6806fa80 con 0x7f8b68132d40 2024-09-13T18:36:20.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 0x7f8b6806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 0x7f8b6806de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47156/0 (socket says 172.21.15.177:47156) 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 -- 172.21.15.177:0/1024763217 learned_addr learned my addr 172.21.15.177:0/1024763217 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b66d9d640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b68132d40 0x7f8b6806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b65d9b640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6806bf30 0x7f8b6806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 -- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b68132d40 msgr2=0x7f8b6806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b68132d40 0x7f8b6806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 -- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6806bf30 msgr2=0x7f8b6806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6806bf30 0x7f8b6806e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 -- 172.21.15.177:0/1024763217 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009a70 con 0x7f8b6806b440 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b66d9d640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b68132d40 0x7f8b6806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b65d9b640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6806bf30 0x7f8b6806e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6659c640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 0x7f8b6806de20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c4a0 tx=0x7f8b5800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:20.784 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580074e0 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b58007640 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6806f130 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.779+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004a00 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.780+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b681321f0 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.781+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f8b58004b60 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.781+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b681324d0 con 0x7f8b6806b440 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.781+0000 7f8b65d9b640 1 -- 172.21.15.177:0/1024763217 >> v1:172.21.15.177:6816/32265 conn(0x7f8b4804cff0 legacy=0x7f8b4804f4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:20.785 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.781+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f8b580627e0 con 0x7f8b6806b440 2024-09-13T18:36:20.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.785+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f8b5803d640 con 0x7f8b6806b440 2024-09-13T18:36:20.839 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.837+0000 7eff70123040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:36:20.842 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.840+0000 7f516140c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:36:20.887 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:20.885+0000 7eff70123040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:36:20.887 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:20.885+0000 7f516140c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:36:20.925 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.923+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b680704b0 con 0x7f8b6806b440 2024-09-13T18:36:20.925 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.923+0000 7f8b56ffd640 1 -- 172.21.15.177:0/1024763217 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+8628 (secure 0 0 0) 0x7f8b58018020 con 0x7f8b6806b440 2024-09-13T18:36:20.926 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:20.926 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":19,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:35:18.982537+0000","last_up_change":"2024-09-13T18:35:03.616896+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":32756}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":32756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":32756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":32756}]},"public_addr":"172.21.15.177:6804/32756","cluster_addr":"172.21.15.177:6806/32756","heartbeat_back_addr":"172.21.15.177:6810/32756","heartbeat_front_addr":"172.21.15.177:6809/32756","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 >> v1:172.21.15.177:6816/32265 conn(0x7f8b4804cff0 legacy=0x7f8b4804f4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 msgr2=0x7f8b6806de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 0x7f8b6806de20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c4a0 tx=0x7f8b5800c960 comp rx=0 tx=0).stop 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 shutdown_connections 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8b68132d40 0x7f8b6806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6806bf30 0x7f8b6806e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 --2- 172.21.15.177:0/1024763217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6806b440 0x7f8b6806de20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 >> 172.21.15.177:0/1024763217 conn(0x7f8b6809fe10 msgr2=0x7f8b6809bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 shutdown_connections 2024-09-13T18:36:20.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:20.926+0000 7f8b6cb45640 1 -- 172.21.15.177:0/1024763217 wait complete. 2024-09-13T18:36:20.944 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T18:36:20.944 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T18:36:20.944 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T18:36:20.945 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T18:36:21.054 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.052+0000 7f93f84aa640 1 Processor -- start 2024-09-13T18:36:21.055 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.052+0000 7feb4b31f640 1 Processor -- start 2024-09-13T18:36:21.055 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.052+0000 7f93f84aa640 1 -- start start 2024-09-13T18:36:21.055 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.052+0000 7feb4b31f640 1 -- start start 2024-09-13T18:36:21.056 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f84aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f006b440 0x7f93f006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.056 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f84aa640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f012dbf0 0x7f93f012e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.056 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f84aa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f006c4a0 con 0x7f93f006bd60 2024-09-13T18:36:21.056 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f84aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f006d900 con 0x7f93f006b440 2024-09-13T18:36:21.056 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f84aa640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93f006db60 con 0x7f93f012dbf0 2024-09-13T18:36:21.057 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb4b31f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feb4406bcc0 0x7feb44130770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.057 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb4b31f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb44130cb0 0x7feb441330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.057 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb4b31f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4406fb40 con 0x7feb4406b1d0 2024-09-13T18:36:21.057 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f5a45640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f93f006bd60 0x7f93f012d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58090/0 (socket says 172.21.15.177:58090) 2024-09-13T18:36:21.057 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f5a45640 1 -- 172.21.15.177:0/2827349519 learned_addr learned my addr 172.21.15.177:0/2827349519 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:21.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb4b31f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4406c0a0 con 0x7feb4406bcc0 2024-09-13T18:36:21.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb4b31f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb44097e00 con 0x7feb44130cb0 2024-09-13T18:36:21.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb490bb640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7feb4406b1d0 0x7feb4406b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58080/0 (socket says 172.21.15.177:58080) 2024-09-13T18:36:21.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb490bb640 1 -- 172.21.15.177:0/2096383609 learned_addr learned my addr 172.21.15.177:0/2096383609 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:21.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7feb498bc640 1 --2- 172.21.15.177:0/2096383609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb44130cb0 0x7feb441330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.053+0000 7f93f6a47640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f012dbf0 0x7f93f012e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb488ba640 1 --2- 172.21.15.177:0/2096383609 >> v2:172.21.15.177:3300/0 conn(0x7feb4406bcc0 0x7feb44130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb37fff640 1 -- 172.21.15.177:0/2096383609 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198243662 0 0) 0x7feb4406fb40 con 0x7feb4406b1d0 2024-09-13T18:36:21.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb37fff640 1 -- 172.21.15.177:0/2096383609 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb30003520 con 0x7feb4406b1d0 2024-09-13T18:36:21.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7f93f6a47640 1 -- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f006b440 msgr2=0x7f93f006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb498bc640 1 -- 172.21.15.177:0/2096383609 >> v2:172.21.15.177:3300/0 conn(0x7feb4406bcc0 msgr2=0x7feb44130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb498bc640 1 --2- 172.21.15.177:0/2096383609 >> v2:172.21.15.177:3300/0 conn(0x7feb4406bcc0 0x7feb44130770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb498bc640 1 -- 172.21.15.177:0/2096383609 >> v1:172.21.15.177:6789/0 conn(0x7feb4406b1d0 legacy=0x7feb4406b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7feb498bc640 1 -- 172.21.15.177:0/2096383609 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4406f1c0 con 0x7feb44130cb0 2024-09-13T18:36:21.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7f93f6246640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f006b440 0x7f93f006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7f93f5244640 1 -- 172.21.15.177:0/2827349519 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013396825 0 0) 0x7f93f006c4a0 con 0x7f93f006bd60 2024-09-13T18:36:21.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7f93f6a47640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f006b440 0x7f93f006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7f93f6a47640 1 -- 172.21.15.177:0/2827349519 >> v1:172.21.15.177:6789/0 conn(0x7f93f006bd60 legacy=0x7f93f012d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.054+0000 7f93f6a47640 1 -- 172.21.15.177:0/2827349519 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f006c140 con 0x7f93f012dbf0 2024-09-13T18:36:21.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.055+0000 7f93f6246640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f006b440 0x7f93f006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:21.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.055+0000 7feb498bc640 1 --2- 172.21.15.177:0/2096383609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb44130cb0 0x7feb441330a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009d90 tx=0x7feb3c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0ebf27fe177637 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.055+0000 7feb37fff640 1 -- 172.21.15.177:0/2096383609 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c007740 con 0x7feb44130cb0 2024-09-13T18:36:21.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.055+0000 7feb37fff640 1 -- 172.21.15.177:0/2096383609 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c02d050 con 0x7feb44130cb0 2024-09-13T18:36:21.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.055+0000 7feb37fff640 1 -- 172.21.15.177:0/2096383609 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c007aa0 con 0x7feb44130cb0 2024-09-13T18:36:21.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.055+0000 7f93f6a47640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f012dbf0 0x7f93f012e060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b080 tx=0x7f93e802a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f6fd409afe97ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7f93f5244640 1 -- 172.21.15.177:0/2827349519 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8031600 con 0x7f93f012dbf0 2024-09-13T18:36:21.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7f93f5244640 1 -- 172.21.15.177:0/2827349519 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e802d050 con 0x7f93f012dbf0 2024-09-13T18:36:21.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7f93f5244640 1 -- 172.21.15.177:0/2827349519 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8031980 con 0x7f93f012dbf0 2024-09-13T18:36:21.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 -- 172.21.15.177:0/2096383609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb44130cb0 msgr2=0x7feb441330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 --2- 172.21.15.177:0/2096383609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb44130cb0 0x7feb441330a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009d90 tx=0x7feb3c02a530 comp rx=0 tx=0).stop 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 -- 172.21.15.177:0/2096383609 shutdown_connections 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 --2- 172.21.15.177:0/2096383609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb44130cb0 0x7feb441330a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 --2- 172.21.15.177:0/2096383609 >> v2:172.21.15.177:3300/0 conn(0x7feb4406bcc0 0x7feb44130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 -- 172.21.15.177:0/2096383609 >> 172.21.15.177:0/2096383609 conn(0x7feb44098ba0 msgr2=0x7feb44096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 -- 172.21.15.177:0/2096383609 shutdown_connections 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7f93f84aa640 1 -- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f012dbf0 msgr2=0x7f93f012e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.064 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7f93f84aa640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f012dbf0 0x7f93f012e060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b080 tx=0x7f93e802a770 comp rx=0 tx=0).stop 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.056+0000 7feb4b31f640 1 -- 172.21.15.177:0/2096383609 wait complete. 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7f93f84aa640 1 -- 172.21.15.177:0/2827349519 shutdown_connections 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7f93f84aa640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f012dbf0 0x7f93f012e060 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7f93f84aa640 1 --2- 172.21.15.177:0/2827349519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f006b440 0x7f93f006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7f93f84aa640 1 -- 172.21.15.177:0/2827349519 >> 172.21.15.177:0/2827349519 conn(0x7f93f009fe10 msgr2=0x7f93f00a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7f93f84aa640 1 -- 172.21.15.177:0/2827349519 shutdown_connections 2024-09-13T18:36:21.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7f93f84aa640 1 -- 172.21.15.177:0/2827349519 wait complete. 2024-09-13T18:36:21.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7feb4b31f640 1 Processor -- start 2024-09-13T18:36:21.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.057+0000 7feb4b31f640 1 -- start start 2024-09-13T18:36:21.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 Processor -- start 2024-09-13T18:36:21.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 -- start start 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb4b31f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb4406b1d0 0x7feb441c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb4b31f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7feb4406bcc0 0x7feb441c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb4b31f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb44130cb0 0x7feb44127810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb4b31f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44093d10 con 0x7feb44130cb0 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb4b31f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb44095c40 con 0x7feb4406b1d0 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb4b31f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb44094170 con 0x7feb4406bcc0 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb490bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb4406b1d0 0x7feb441c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb490bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb4406b1d0 0x7feb441c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35036/0 (socket says 172.21.15.177:35036) 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb490bb640 1 -- 172.21.15.177:0/997166440 learned_addr learned my addr 172.21.15.177:0/997166440 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7feb488ba640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7feb4406bcc0 0x7feb441c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93f006b440 0x7f93f0099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f006bd60 0x7f93f0096680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f012dbf0 0x7f93f0096bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00702f0 con 0x7f93f006b440 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0099290 con 0x7f93f012dbf0 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.058+0000 7f93f84aa640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0098fd0 con 0x7f93f006bd60 2024-09-13T18:36:21.069 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f6246640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93f006b440 0x7f93f0099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb498bc640 1 --2- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb44130cb0 0x7feb44127810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f6a47640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f012dbf0 0x7f93f0096bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f006bd60 0x7f93f0096680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f6a47640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f012dbf0 0x7f93f0096bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35050/0 (socket says 172.21.15.177:35050) 2024-09-13T18:36:21.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f6a47640 1 -- 172.21.15.177:0/4140106650 learned_addr learned my addr 172.21.15.177:0/4140106650 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb488ba640 1 -- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb4406b1d0 msgr2=0x7feb441c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb488ba640 1 --2- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb4406b1d0 0x7feb441c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb488ba640 1 -- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb44130cb0 msgr2=0x7feb44127810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb488ba640 1 --2- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb44130cb0 0x7feb44127810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb488ba640 1 -- 172.21.15.177:0/997166440 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c009a70 con 0x7feb4406bcc0 2024-09-13T18:36:21.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 -- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f012dbf0 msgr2=0x7f93f0096bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 --2- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93f012dbf0 0x7f93f0096bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 -- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93f006b440 msgr2=0x7f93f0099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 --2- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93f006b440 0x7f93f0099ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 -- 172.21.15.177:0/4140106650 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e8009a70 con 0x7f93f006bd60 2024-09-13T18:36:21.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f6246640 1 --2- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93f006b440 0x7f93f0099ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:21.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7feb488ba640 1 --2- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7feb4406bcc0 0x7feb441c6f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb3800e850 tx=0x7feb3800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.059+0000 7f93f5a45640 1 --2- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f006bd60 0x7f93f0096680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c450 tx=0x7f93e000c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb380095b0 con 0x7feb4406bcc0 2024-09-13T18:36:21.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00074e0 con 0x7f93f006bd60 2024-09-13T18:36:21.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb440955b0 con 0x7feb4406bcc0 2024-09-13T18:36:21.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb44127d50 con 0x7feb4406bcc0 2024-09-13T18:36:21.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f0097bf0 con 0x7f93f006bd60 2024-09-13T18:36:21.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f0097100 con 0x7f93f006bd60 2024-09-13T18:36:21.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.060+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38005990 con 0x7feb4406bcc0 2024-09-13T18:36:21.075 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.061+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93f0070ca0 con 0x7f93f006bd60 2024-09-13T18:36:21.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.061+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38005d00 con 0x7feb4406bcc0 2024-09-13T18:36:21.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.061+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0007640 con 0x7f93f006bd60 2024-09-13T18:36:21.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.061+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0004a20 con 0x7f93f006bd60 2024-09-13T18:36:21.076 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.062+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb24000fc0 con 0x7feb4406bcc0 2024-09-13T18:36:21.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.062+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f93e0004b80 con 0x7f93f006bd60 2024-09-13T18:36:21.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.062+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7feb3800c940 con 0x7feb4406bcc0 2024-09-13T18:36:21.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7f93f6246640 1 -- 172.21.15.177:0/4140106650 >> v1:172.21.15.177:6816/32265 conn(0x7f93d404cfc0 legacy=0x7f93d404f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.078 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7feb490bb640 1 -- 172.21.15.177:0/997166440 >> v1:172.21.15.177:6816/32265 conn(0x7feb2804cfc0 legacy=0x7feb2804f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.079 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7feb38063460 con 0x7feb4406bcc0 2024-09-13T18:36:21.080 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f93e0063040 con 0x7f93f006bd60 2024-09-13T18:36:21.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 --> v1:172.21.15.177:6812/32757 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93d4057320 con 0x7f93d4053650 2024-09-13T18:36:21.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e0063490 con 0x7f93f006bd60 2024-09-13T18:36:21.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 --> v1:172.21.15.177:6800/32755 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb280571b0 con 0x7feb280534e0 2024-09-13T18:36:21.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.063+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb38065af0 con 0x7feb4406bcc0 2024-09-13T18:36:21.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.064+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== osd.3 v1:172.21.15.177:6800/32755 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7feb280571b0 con 0x7feb280534e0 2024-09-13T18:36:21.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.064+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== osd.1 v1:172.21.15.177:6812/32757 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f93d4057320 con 0x7f93d4053650 2024-09-13T18:36:21.092 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.090+0000 7f516140c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:36:21.092 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.090+0000 7eff70123040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:36:21.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.093+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 --> v1:172.21.15.177:6812/32757 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93f0097f40 con 0x7f93d4053650 2024-09-13T18:36:21.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.093+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 --> v1:172.21.15.177:6800/32755 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feb24002c80 con 0x7feb280534e0 2024-09-13T18:36:21.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.093+0000 7feb35ffb640 1 -- 172.21.15.177:0/997166440 <== osd.3 v1:172.21.15.177:6800/32755 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3077695196) 0x7feb24002c80 con 0x7feb280534e0 2024-09-13T18:36:21.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.093+0000 7f93e6ffd640 1 -- 172.21.15.177:0/4140106650 <== osd.1 v1:172.21.15.177:6812/32757 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1159357919) 0x7f93f0097f40 con 0x7f93d4053650 2024-09-13T18:36:21.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.093+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 >> v1:172.21.15.177:6800/32755 conn(0x7feb280534e0 legacy=0x7feb28055940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.093+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 >> v1:172.21.15.177:6812/32757 conn(0x7f93d4053650 legacy=0x7f93d4055ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 >> v1:172.21.15.177:6816/32265 conn(0x7feb2804cfc0 legacy=0x7feb2804f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7feb4406bcc0 msgr2=0x7feb441c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 --2- 172.21.15.177:0/997166440 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7feb4406bcc0 0x7feb441c6f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb3800e850 tx=0x7feb3800ed10 comp rx=0 tx=0).stop 2024-09-13T18:36:21.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 >> v1:172.21.15.177:6816/32265 conn(0x7f93d404cfc0 legacy=0x7f93d404f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb498bc640 1 -- 172.21.15.177:0/997166440 reap_dead start 2024-09-13T18:36:21.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f006bd60 msgr2=0x7f93f0096680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 --2- 172.21.15.177:0/4140106650 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f93f006bd60 0x7f93f0096680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c450 tx=0x7f93e000c910 comp rx=0 tx=0).stop 2024-09-13T18:36:21.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f6a47640 1 -- 172.21.15.177:0/4140106650 reap_dead start 2024-09-13T18:36:21.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 shutdown_connections 2024-09-13T18:36:21.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 >> 172.21.15.177:0/4140106650 conn(0x7f93f009fe10 msgr2=0x7f93f009b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 shutdown_connections 2024-09-13T18:36:21.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 >> 172.21.15.177:0/997166440 conn(0x7feb44098ba0 msgr2=0x7feb4409b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 shutdown_connections 2024-09-13T18:36:21.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7f93f84aa640 1 -- 172.21.15.177:0/4140106650 wait complete. 2024-09-13T18:36:21.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 shutdown_connections 2024-09-13T18:36:21.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.094+0000 7feb4b31f640 1 -- 172.21.15.177:0/997166440 wait complete. 2024-09-13T18:36:21.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.098+0000 7ff7bad08640 1 Processor -- start 2024-09-13T18:36:21.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.098+0000 7ff7bad08640 1 -- start start 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7bad08640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 0x7ff7b4127630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7bad08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412a2b0 0x7ff7b412a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7b8aa4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 0x7ff7b4127630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7b8aa4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 0x7ff7b4127630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60950/0 (socket says 172.21.15.177:60950) 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/4059407905 learned_addr learned my addr 172.21.15.177:0/4059407905 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7b3fff640 1 --2- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412a2b0 0x7ff7b412a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40967c0 con 0x7ff7b412ac50 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b412b100 con 0x7ff7b412a2b0 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.099+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4096200 con 0x7ff7b4127250 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7ff7b37fe640 1 -- 172.21.15.177:0/4059407905 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516428401 0 0) 0x7ff7b40967c0 con 0x7ff7b412ac50 2024-09-13T18:36:21.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7ff7b37fe640 1 -- 172.21.15.177:0/4059407905 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a4003520 con 0x7ff7b412ac50 2024-09-13T18:36:21.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412a2b0 msgr2=0x7ff7b412a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7ff7b8aa4640 1 --2- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412a2b0 0x7ff7b412a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/4059407905 >> v1:172.21.15.177:6789/0 conn(0x7ff7b412ac50 legacy=0x7ff7b4131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/4059407905 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4096a60 con 0x7ff7b4127250 2024-09-13T18:36:21.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.100+0000 7f9ccae9c640 1 Processor -- start 2024-09-13T18:36:21.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7f9ccae9c640 1 -- start start 2024-09-13T18:36:21.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7ff7b8aa4640 1 --2- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 0x7ff7b4127630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80098a0 tx=0x7ff7a802c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad949a5db7f536c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7f9ccae9c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc4127250 0x7f9cc4127630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7f9ccae9c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9cc412ae20 0x7f9cc4132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7ff7b37fe640 1 -- 172.21.15.177:0/4059407905 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8034640 con 0x7ff7b4127250 2024-09-13T18:36:21.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7f9ccae9c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc40967c0 con 0x7f9cc412a2b0 2024-09-13T18:36:21.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7f9ccae9c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc4096200 con 0x7f9cc412ae20 2024-09-13T18:36:21.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7f9ccae9c640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4096a60 con 0x7f9cc4127250 2024-09-13T18:36:21.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7ff7b37fe640 1 -- 172.21.15.177:0/4059407905 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a80347a0 con 0x7ff7b4127250 2024-09-13T18:36:21.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.101+0000 7ff7b37fe640 1 -- 172.21.15.177:0/4059407905 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8034b10 con 0x7ff7b4127250 2024-09-13T18:36:21.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc3fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f9cc412a2b0 0x7f9cc412a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58112/0 (socket says 172.21.15.177:58112) 2024-09-13T18:36:21.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc3fff640 1 -- 172.21.15.177:0/3610042181 learned_addr learned my addr 172.21.15.177:0/3610042181 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:21.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc8c38640 1 --2- 172.21.15.177:0/3610042181 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc4127250 0x7f9cc4127630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc9439640 1 --2- 172.21.15.177:0/3610042181 >> v2:172.21.15.177:3300/0 conn(0x7f9cc412ae20 0x7f9cc4132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc8c38640 1 -- 172.21.15.177:0/3610042181 >> v2:172.21.15.177:3300/0 conn(0x7f9cc412ae20 msgr2=0x7f9cc4132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc8c38640 1 --2- 172.21.15.177:0/3610042181 >> v2:172.21.15.177:3300/0 conn(0x7f9cc412ae20 0x7f9cc4132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc8c38640 1 -- 172.21.15.177:0/3610042181 >> v1:172.21.15.177:6789/0 conn(0x7f9cc412a2b0 legacy=0x7f9cc412a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc8c38640 1 -- 172.21.15.177:0/3610042181 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4096d20 con 0x7f9cc4127250 2024-09-13T18:36:21.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7f9cc9439640 1 --2- 172.21.15.177:0/3610042181 >> v2:172.21.15.177:3300/0 conn(0x7f9cc412ae20 0x7f9cc4132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 msgr2=0x7ff7b4127630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.102+0000 7ff7bad08640 1 --2- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 0x7ff7b4127630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80098a0 tx=0x7ff7a802c3b0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 shutdown_connections 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7ff7bad08640 1 --2- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412a2b0 0x7ff7b412a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7ff7bad08640 1 --2- 172.21.15.177:0/4059407905 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b4127250 0x7ff7b4127630 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 >> 172.21.15.177:0/4059407905 conn(0x7ff7b409cd60 msgr2=0x7ff7b409d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 shutdown_connections 2024-09-13T18:36:21.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7ff7bad08640 1 -- 172.21.15.177:0/4059407905 wait complete. 2024-09-13T18:36:21.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.103+0000 7f9cc8c38640 1 --2- 172.21.15.177:0/3610042181 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc4127250 0x7f9cc4127630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009a00 tx=0x7f9cb802a350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a52a61106df62a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.104+0000 7f9cc37fe640 1 -- 172.21.15.177:0/3610042181 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb802ae90 con 0x7f9cc4127250 2024-09-13T18:36:21.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.104+0000 7f9cc37fe640 1 -- 172.21.15.177:0/3610042181 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8032440 con 0x7f9cc4127250 2024-09-13T18:36:21.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.104+0000 7f9cc37fe640 1 -- 172.21.15.177:0/3610042181 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80327d0 con 0x7f9cc4127250 2024-09-13T18:36:21.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.104+0000 7ff7bad08640 1 Processor -- start 2024-09-13T18:36:21.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.104+0000 7ff7bad08640 1 -- start start 2024-09-13T18:36:21.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7bad08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b4127250 0x7ff7b406dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 -- 172.21.15.177:0/3610042181 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc4127250 msgr2=0x7f9cc4127630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 --2- 172.21.15.177:0/3610042181 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc4127250 0x7f9cc4127630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009a00 tx=0x7f9cb802a350 comp rx=0 tx=0).stop 2024-09-13T18:36:21.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7bad08640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b412a2b0 0x7ff7b406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7bad08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412ac50 0x7ff7b406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7bad08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b406fb60 con 0x7ff7b4127250 2024-09-13T18:36:21.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7bad08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4070b90 con 0x7ff7b412ac50 2024-09-13T18:36:21.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7bad08640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b406efa0 con 0x7ff7b412a2b0 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b4127250 0x7ff7b406dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b4127250 0x7ff7b406dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47202/0 (socket says 172.21.15.177:47202) 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/1480373396 learned_addr learned my addr 172.21.15.177:0/1480373396 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b92a5640 1 --2- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412ac50 0x7ff7b406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b3fff640 1 --2- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b412a2b0 0x7ff7b406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 -- 172.21.15.177:0/3610042181 shutdown_connections 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 --2- 172.21.15.177:0/3610042181 >> v2:172.21.15.177:3300/0 conn(0x7f9cc412ae20 0x7f9cc4132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 --2- 172.21.15.177:0/3610042181 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc4127250 0x7f9cc4127630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 -- 172.21.15.177:0/3610042181 >> 172.21.15.177:0/3610042181 conn(0x7f9cc409cd60 msgr2=0x7f9cc409d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 -- 172.21.15.177:0/3610042181 shutdown_connections 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b412a2b0 msgr2=0x7ff7b406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 --2- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ff7b412a2b0 0x7ff7b406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412ac50 msgr2=0x7ff7b406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 --2- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b412ac50 0x7ff7b406ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7ff7b8aa4640 1 -- 172.21.15.177:0/1480373396 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a80094d0 con 0x7ff7b4127250 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.106+0000 7ff7b8aa4640 1 --2- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b4127250 0x7ff7b406dfe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000c00 tx=0x7ff7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.105+0000 7f9ccae9c640 1 -- 172.21.15.177:0/3610042181 wait complete. 2024-09-13T18:36:21.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.106+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8003a20 con 0x7ff7b4127250 2024-09-13T18:36:21.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.106+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a8003b80 con 0x7ff7b4127250 2024-09-13T18:36:21.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.106+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b406f740 con 0x7ff7b4127250 2024-09-13T18:36:21.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.106+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a803f460 con 0x7ff7b4127250 2024-09-13T18:36:21.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.107+0000 7f9ccae9c640 1 Processor -- start 2024-09-13T18:36:21.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.106+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b41c84d0 con 0x7ff7b4127250 2024-09-13T18:36:21.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.107+0000 7f9ccae9c640 1 -- start start 2024-09-13T18:36:21.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.107+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7b40709b0 con 0x7ff7b4127250 2024-09-13T18:36:21.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.107+0000 7f9ccae9c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cc4127250 0x7f9cc41be290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.107+0000 7f9ccae9c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc412a2b0 0x7f9cc41be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9ccae9c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cc412ae20 0x7f9cc406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9cc3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc412a2b0 0x7f9cc41be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9cc8c38640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cc4127250 0x7f9cc41be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9cc8c38640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cc4127250 0x7f9cc41be290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47218/0 (socket says 172.21.15.177:47218) 2024-09-13T18:36:21.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9cc3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc412a2b0 0x7f9cc41be7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60992/0 (socket says 172.21.15.177:60992) 2024-09-13T18:36:21.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7ff7a803e3c0 con 0x7ff7b4127250 2024-09-13T18:36:21.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9cc8c38640 1 -- 172.21.15.177:0/404792887 learned_addr learned my addr 172.21.15.177:0/404792887 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9cc9439640 1 --2- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cc412ae20 0x7f9cc406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9ccae9c640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc408b0f0 con 0x7f9cc4127250 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9ccae9c640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc408b4a0 con 0x7f9cc412ae20 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.108+0000 7f9ccae9c640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40959e0 con 0x7f9cc412a2b0 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7ff7a8084540 con 0x7ff7b4127250 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc8c38640 1 -- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc412a2b0 msgr2=0x7f9cc41be7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc8c38640 1 --2- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9cc412a2b0 0x7f9cc41be7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc8c38640 1 -- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cc412ae20 msgr2=0x7f9cc406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 --> v1:172.21.15.177:6804/32756 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7980573f0 con 0x7ff798053720 2024-09-13T18:36:21.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7a80848f0 con 0x7ff7b4127250 2024-09-13T18:36:21.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7ff7b3fff640 1 -- 172.21.15.177:0/1480373396 >> v1:172.21.15.177:6816/32265 conn(0x7ff79804d090 legacy=0x7ff79804f550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc8c38640 1 --2- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cc412ae20 0x7f9cc406de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc8c38640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8009400 con 0x7f9cc4127250 2024-09-13T18:36:21.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc9439640 1 --2- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cc412ae20 0x7f9cc406de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:21.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.109+0000 7f9cc8c38640 1 --2- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cc4127250 0x7f9cc41be290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8035040 tx=0x7f9cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.110+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80327b0 con 0x7f9cc4127250 2024-09-13T18:36:21.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.110+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8032910 con 0x7f9cc4127250 2024-09-13T18:36:21.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.110+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8032cc0 con 0x7f9cc4127250 2024-09-13T18:36:21.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.110+0000 7f9ccae9c640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc406f9f0 con 0x7f9cc4127250 2024-09-13T18:36:21.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.110+0000 7f9ccae9c640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc40700f0 con 0x7f9cc4127250 2024-09-13T18:36:21.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.111+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== osd.0 v1:172.21.15.177:6804/32756 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7ff7980573f0 con 0x7ff798053720 2024-09-13T18:36:21.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.111+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9cc4127b70 con 0x7f9cc4127250 2024-09-13T18:36:21.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.112+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f9cb803e450 con 0x7f9cc4127250 2024-09-13T18:36:21.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.112+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f9cb8084390 con 0x7f9cc4127250 2024-09-13T18:36:21.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.112+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 --> v1:172.21.15.177:6805/32758 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ca8057330 con 0x7f9ca8053660 2024-09-13T18:36:21.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.112+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb8086970 con 0x7f9cc4127250 2024-09-13T18:36:21.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.112+0000 7f9cc3fff640 1 -- 172.21.15.177:0/404792887 >> v1:172.21.15.177:6816/32265 conn(0x7f9ca804cfd0 legacy=0x7f9ca804f490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.125 INFO:teuthology.orchestra.run.smithi177.stdout:38654705683 2024-09-13T18:36:21.125 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:21.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.114+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== osd.2 v1:172.21.15.177:6805/32758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f9cbc014e60 con 0x7f9ca8053660 2024-09-13T18:36:21.126 INFO:teuthology.orchestra.run.smithi177.stdout:38654705682 2024-09-13T18:36:21.127 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:21.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.137+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 --> v1:172.21.15.177:6804/32756 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7b40709b0 con 0x7ff798053720 2024-09-13T18:36:21.141 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.138+0000 7ff7b17fa640 1 -- 172.21.15.177:0/1480373396 <== osd.0 v1:172.21.15.177:6804/32756 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3077695196) 0x7ff7b40709b0 con 0x7ff798053720 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.138+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 >> v1:172.21.15.177:6804/32756 conn(0x7ff798053720 legacy=0x7ff798055b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.138+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 >> v1:172.21.15.177:6816/32265 conn(0x7ff79804d090 legacy=0x7ff79804f550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.138+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b4127250 msgr2=0x7ff7b406dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.138+0000 7ff7bad08640 1 --2- 172.21.15.177:0/1480373396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b4127250 0x7ff7b406dfe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000c00 tx=0x7ff7a80046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.138+0000 7ff7b92a5640 1 -- 172.21.15.177:0/1480373396 reap_dead start 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.139+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 shutdown_connections 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.139+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 >> 172.21.15.177:0/1480373396 conn(0x7ff7b409cd60 msgr2=0x7ff7b4097fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.139+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 shutdown_connections 2024-09-13T18:36:21.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.139+0000 7ff7bad08640 1 -- 172.21.15.177:0/1480373396 wait complete. 2024-09-13T18:36:21.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.140+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 --> v1:172.21.15.177:6805/32758 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9cc4070a20 con 0x7f9ca8053660 2024-09-13T18:36:21.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.140+0000 7f9cc17fa640 1 -- 172.21.15.177:0/404792887 <== osd.2 v1:172.21.15.177:6805/32758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1159357919) 0x7f9cc4070a20 con 0x7f9ca8053660 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 >> v1:172.21.15.177:6805/32758 conn(0x7f9ca8053660 legacy=0x7f9ca8055ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 >> v1:172.21.15.177:6816/32265 conn(0x7f9ca804cfd0 legacy=0x7f9ca804f490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cc4127250 msgr2=0x7f9cc41be290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9caeffd640 1 --2- 172.21.15.177:0/404792887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cc4127250 0x7f9cc41be290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8035040 tx=0x7f9cb80046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9cc9439640 1 -- 172.21.15.177:0/404792887 reap_dead start 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 shutdown_connections 2024-09-13T18:36:21.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.141+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 >> 172.21.15.177:0/404792887 conn(0x7f9cc409cd60 msgr2=0x7f9cc4097580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.142+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 shutdown_connections 2024-09-13T18:36:21.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.142+0000 7f9caeffd640 1 -- 172.21.15.177:0/404792887 wait complete. 2024-09-13T18:36:21.156 INFO:teuthology.orchestra.run.smithi177.stdout:38654705683 2024-09-13T18:36:21.156 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:21.163 INFO:teuthology.orchestra.run.smithi177.stdout:38654705682 2024-09-13T18:36:21.163 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:21.256 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.253+0000 7f4ba5b18640 1 Processor -- start 2024-09-13T18:36:21.256 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.254+0000 7f4ba5b18640 1 -- start start 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.254+0000 7f4ba5b18640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ba006c590 0x7f4ba006c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.254+0000 7f4ba5b18640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba012a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba012a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4ba5b18640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba006ea10 con 0x7f4ba012a770 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ba006c590 0x7f4ba006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4ba5b18640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba006e230 con 0x7f4ba006c590 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4ba5b18640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba006d2f0 con 0x7f4ba006ceb0 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f4ba012a770 0x7f4ba012abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58118/0 (socket says 172.21.15.177:58118) 2024-09-13T18:36:21.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9ffff640 1 -- 172.21.15.177:0/1375211068 learned_addr learned my addr 172.21.15.177:0/1375211068 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:21.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9f7fe640 1 -- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 msgr2=0x7f4ba012a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9f7fe640 1 --2- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba012a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9f7fe640 1 -- 172.21.15.177:0/1375211068 >> v1:172.21.15.177:6789/0 conn(0x7f4ba012a770 legacy=0x7f4ba012abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.255+0000 7f4b9f7fe640 1 -- 172.21.15.177:0/1375211068 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba006ede0 con 0x7f4ba006c590 2024-09-13T18:36:21.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.256+0000 7f4b9effd640 1 --2- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba012a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:21.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.256+0000 7f74c9dc7640 1 Processor -- start 2024-09-13T18:36:21.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.256+0000 7f74c9dc7640 1 -- start start 2024-09-13T18:36:21.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.256+0000 7f4b9f7fe640 1 --2- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba006c590 0x7f4ba006c970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b900097a0 tx=0x7f4b9002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc8f33a848de250 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c9dc7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 0x7f74c4095070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c37fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 0x7f74c4095070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c37fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 0x7f74c4095070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47240/0 (socket says 172.21.15.177:47240) 2024-09-13T18:36:21.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c9dc7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c40955b0 0x7f74c4091fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f4b9e7fc640 1 -- 172.21.15.177:0/1375211068 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9002adb0 con 0x7f4ba006c590 2024-09-13T18:36:21.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f4b9e7fc640 1 -- 172.21.15.177:0/1375211068 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90032440 con 0x7f4ba006c590 2024-09-13T18:36:21.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f4b9e7fc640 1 -- 172.21.15.177:0/1375211068 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b900327d0 con 0x7f4ba006c590 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c37fe640 1 -- 172.21.15.177:0/791851097 learned_addr learned my addr 172.21.15.177:0/791851097 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c40929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c2ffd640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c40955b0 0x7f74c4091fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c3fff640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c40929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c4072240 con 0x7f74c4094c90 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40939a0 con 0x7f74c40955b0 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.257+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74c406bd10 con 0x7f74c4092510 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c2ffd640 1 -- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 msgr2=0x7f74c40929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c2ffd640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c40929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c2ffd640 1 -- 172.21.15.177:0/791851097 >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 msgr2=0x7f74c4095070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c2ffd640 1 --2- 172.21.15.177:0/791851097 >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 0x7f74c4095070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c2ffd640 1 -- 172.21.15.177:0/791851097 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c40713c0 con 0x7f74c40955b0 2024-09-13T18:36:21.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c3fff640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c40929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:21.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba006c590 msgr2=0x7f4ba006c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c37fe640 1 --2- 172.21.15.177:0/791851097 >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 0x7f74c4095070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:21.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba006c590 0x7f4ba006c970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b900097a0 tx=0x7f4b9002a060 comp rx=0 tx=0).stop 2024-09-13T18:36:21.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1375211068 shutdown_connections 2024-09-13T18:36:21.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba012a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1375211068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba006c590 0x7f4ba006c970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1375211068 >> 172.21.15.177:0/1375211068 conn(0x7f4ba009ccd0 msgr2=0x7f4ba009d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.265 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.258+0000 7f74c2ffd640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c40955b0 0x7f74c4091fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009790 tx=0x7f74b8029fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c14dc732af0438 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.265 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.259+0000 7f74c27fc640 1 -- 172.21.15.177:0/791851097 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b802ad10 con 0x7f74c40955b0 2024-09-13T18:36:21.265 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.259+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1375211068 shutdown_connections 2024-09-13T18:36:21.265 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.259+0000 7f74c27fc640 1 -- 172.21.15.177:0/791851097 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b802ae70 con 0x7f74c40955b0 2024-09-13T18:36:21.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.259+0000 7f74c27fc640 1 -- 172.21.15.177:0/791851097 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8032640 con 0x7f74c40955b0 2024-09-13T18:36:21.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.259+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1375211068 wait complete. 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c40955b0 msgr2=0x7f74c4091fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c40955b0 0x7f74c4091fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009790 tx=0x7f74b8029fc0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 shutdown_connections 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c40929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/791851097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c40955b0 0x7f74c4091fd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/791851097 >> v2:172.21.15.177:3300/0 conn(0x7f74c4094c90 0x7f74c4095070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 >> 172.21.15.177:0/791851097 conn(0x7f74c409bfd0 msgr2=0x7f74c409c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 shutdown_connections 2024-09-13T18:36:21.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f74c9dc7640 1 -- 172.21.15.177:0/791851097 wait complete. 2024-09-13T18:36:21.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.260+0000 7f4ba5b18640 1 Processor -- start 2024-09-13T18:36:21.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4ba5b18640 1 -- start start 2024-09-13T18:36:21.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4ba5b18640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 0x7f4ba0128030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4ba5b18640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba0128570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 0x7f4ba0128030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 0x7f4ba0128030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32768/0 (socket says 172.21.15.177:32768) 2024-09-13T18:36:21.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f74c9dc7640 1 Processor -- start 2024-09-13T18:36:21.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4b9effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba0128570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4b9effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba0128570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35090/0 (socket says 172.21.15.177:35090) 2024-09-13T18:36:21.270 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.261+0000 7f4b9f7fe640 1 -- 172.21.15.177:0/1740873093 learned_addr learned my addr 172.21.15.177:0/1740873093 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.270 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba012a770 0x7f4ba0128ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.270 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba008b3b0 con 0x7f4ba012a770 2024-09-13T18:36:21.270 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0129590 con 0x7f4ba006ceb0 2024-09-13T18:36:21.270 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f74c9dc7640 1 -- start start 2024-09-13T18:36:21.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba01291d0 con 0x7f4ba006c590 2024-09-13T18:36:21.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba012a770 0x7f4ba0128ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 -- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 msgr2=0x7f4ba0128030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 0x7f4ba0128030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 -- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 msgr2=0x7f4ba0128570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba0128570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90009400 con 0x7f4ba012a770 2024-09-13T18:36:21.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9effd640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba0128570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:21.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.262+0000 7f4b9ffff640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba012a770 0x7f4ba0128ab0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800ca70 tx=0x7f4b9800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f4b9f7fe640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 0x7f4ba0128030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:21.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f74c9dc7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c4122910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800ab70 con 0x7f4ba012a770 2024-09-13T18:36:21.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800acd0 con 0x7f4ba012a770 2024-09-13T18:36:21.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98012600 con 0x7f4ba012a770 2024-09-13T18:36:21.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f74c37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c4122910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba0131660 con 0x7f4ba012a770 2024-09-13T18:36:21.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba0129be0 con 0x7f4ba012a770 2024-09-13T18:36:21.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f74c37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c4122910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32770/0 (socket says 172.21.15.177:32770) 2024-09-13T18:36:21.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f74c9dc7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 0x7f74c4122e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.263+0000 7f74c2ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 0x7f74c4122e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.264+0000 7f74c2ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 0x7f74c4122e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47250/0 (socket says 172.21.15.177:47250) 2024-09-13T18:36:21.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.264+0000 7f74c37fe640 1 -- 172.21.15.177:0/416358894 learned_addr learned my addr 172.21.15.177:0/416358894 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.264+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba012abe0 con 0x7f4ba012a770 2024-09-13T18:36:21.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.264+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c4125480 0x7f74c4123390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.265+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4124ba0 con 0x7f74c4094c90 2024-09-13T18:36:21.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.265+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4071570 con 0x7f74c4125480 2024-09-13T18:36:21.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.265+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4126280 con 0x7f74c4092510 2024-09-13T18:36:21.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.265+0000 7f74c3fff640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c4125480 0x7f74c4123390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.265+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f4b98012760 con 0x7f4ba012a770 2024-09-13T18:36:21.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f4b98063840 con 0x7f4ba012a770 2024-09-13T18:36:21.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f4b9f7fe640 1 -- 172.21.15.177:0/1740873093 >> v1:172.21.15.177:6816/32265 conn(0x7f4b8c04d090 legacy=0x7f4b8c04f550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c3fff640 1 -- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 msgr2=0x7f74c4122910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c3fff640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c4122910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c3fff640 1 -- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 msgr2=0x7f74c4122e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c3fff640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 0x7f74c4122e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c3fff640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094d0 con 0x7f74c4125480 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c3fff640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c4125480 0x7f74c4123390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74bc007db0 tx=0x7f74bc00d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc015400 con 0x7f74c4125480 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b80093a0 con 0x7f74c4125480 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00f040 con 0x7f74c4125480 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.266+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc015760 con 0x7f74c4125480 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.267+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c4126880 con 0x7f74c4125480 2024-09-13T18:36:21.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.268+0000 7f74c37fe640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c4122910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.268+0000 7f74c2ffd640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 0x7f74c4122e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.268+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c4091fd0 con 0x7f74c4125480 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.268+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f74bc01e3b0 con 0x7f74c4125480 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.272+0000 7f74c37fe640 1 -- 172.21.15.177:0/416358894 >> v1:172.21.15.177:6816/32265 conn(0x7f74b004cfc0 legacy=0x7f74b004f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.272+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f74bc0635c0 con 0x7f74c4125480 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.272+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f4b9803e680 con 0x7f4ba012a770 2024-09-13T18:36:21.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.276+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f74bc03e360 con 0x7f74c4125480 2024-09-13T18:36:21.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.356+0000 7fdb5dbd4640 1 Processor -- start 2024-09-13T18:36:21.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.356+0000 7fdb5dbd4640 1 -- start start 2024-09-13T18:36:21.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb5dbd4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 0x7fdb58136f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb5dbd4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb58137470 0x7fdb58139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb56ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 0x7fdb58136f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb56ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 0x7fdb58136f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35100/0 (socket says 172.21.15.177:35100) 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb57fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb58137470 0x7fdb58139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb577fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fdb58095100 0x7fdb580954e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58122/0 (socket says 172.21.15.177:58122) 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.357+0000 7fdb57fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb58137470 0x7fdb58139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47252/0 (socket says 172.21.15.177:47252) 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb56ffd640 1 -- 172.21.15.177:0/1703055625 learned_addr learned my addr 172.21.15.177:0/1703055625 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb580961b0 con 0x7fdb58095100 2024-09-13T18:36:21.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5809a260 con 0x7fdb58137470 2024-09-13T18:36:21.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58099ca0 con 0x7fdb58095bf0 2024-09-13T18:36:21.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb567fc640 1 -- 172.21.15.177:0/1703055625 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806468909 0 0) 0x7fdb580961b0 con 0x7fdb58095100 2024-09-13T18:36:21.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb57fff640 1 -- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 msgr2=0x7fdb58136f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb57fff640 1 --2- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 0x7fdb58136f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb57fff640 1 -- 172.21.15.177:0/1703055625 >> v1:172.21.15.177:6789/0 conn(0x7fdb58095100 legacy=0x7fdb580954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb57fff640 1 -- 172.21.15.177:0/1703055625 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58096d60 con 0x7fdb58137470 2024-09-13T18:36:21.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.358+0000 7fdb56ffd640 1 --2- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 0x7fdb58136f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:21.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.360+0000 7fdb57fff640 1 --2- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb58139860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009d90 tx=0x7fdb5002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3b6ad1e6d70659 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.360+0000 7fdb567fc640 1 -- 172.21.15.177:0/1703055625 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50031600 con 0x7fdb58137470 2024-09-13T18:36:21.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.360+0000 7fdb567fc640 1 -- 172.21.15.177:0/1703055625 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5002d050 con 0x7fdb58137470 2024-09-13T18:36:21.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.360+0000 7fdb567fc640 1 -- 172.21.15.177:0/1703055625 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50031980 con 0x7fdb58137470 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 msgr2=0x7fdb58139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb58139860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009d90 tx=0x7fdb5002a5d0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 shutdown_connections 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb58139860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/1703055625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095bf0 0x7fdb58136f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 >> 172.21.15.177:0/1703055625 conn(0x7fdb5809ab20 msgr2=0x7fdb5809af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 shutdown_connections 2024-09-13T18:36:21.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.361+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/1703055625 wait complete. 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.362+0000 7fdb5dbd4640 1 Processor -- start 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.363+0000 7fdb5dbd4640 1 -- start start 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.363+0000 7fdb5dbd4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 0x7fdb5812ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.363+0000 7fdb5dbd4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdb58095bf0 0x7fdb5812bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 0x7fdb5812ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 0x7fdb5812ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35104/0 (socket says 172.21.15.177:35104) 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb56ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdb58095bf0 0x7fdb5812bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 -- 172.21.15.177:0/3196708440 learned_addr learned my addr 172.21.15.177:0/3196708440 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb5812c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb57fff640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb5812c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5812de80 con 0x7fdb58137470 2024-09-13T18:36:21.366 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5812ec40 con 0x7fdb58095100 2024-09-13T18:36:21.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5812eea0 con 0x7fdb58095bf0 2024-09-13T18:36:21.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 -- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdb58095bf0 msgr2=0x7fdb5812bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdb58095bf0 0x7fdb5812bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.367 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 -- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 msgr2=0x7fdb5812c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb5812c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.364+0000 7fdb577fe640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0094d0 con 0x7fdb58095100 2024-09-13T18:36:21.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7f5760e12640 1 Processor -- start 2024-09-13T18:36:21.368 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7f5760e12640 1 -- start start 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7fdb577fe640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 0x7fdb5812ba00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c4d0 tx=0x7fdb4800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480074e0 con 0x7fdb58095100 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb48007640 con 0x7fdb58095100 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb50009a70 con 0x7fdb58095100 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7f5760e12640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f575c12e170 0x7f575c132870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7f5760e12640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c132db0 0x7f575c1351a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.369 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7f5760e12640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575c070d10 con 0x7f575c12d550 2024-09-13T18:36:21.370 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.366+0000 7f5760e12640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f575c094200 con 0x7f575c12e170 2024-09-13T18:36:21.370 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.366+0000 7f5760e12640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f575c071530 con 0x7f575c132db0 2024-09-13T18:36:21.370 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48004a20 con 0x7fdb58095100 2024-09-13T18:36:21.370 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.365+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5812e790 con 0x7fdb58095100 2024-09-13T18:36:21.370 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.366+0000 7f5751d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f575c12e170 0x7f575c132870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.370 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.366+0000 7f5751d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f575c12e170 0x7f575c132870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47274/0 (socket says 172.21.15.177:47274) 2024-09-13T18:36:21.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.366+0000 7f5751d9b640 1 -- 172.21.15.177:0/2965880272 learned_addr learned my addr 172.21.15.177:0/2965880272 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.367+0000 7f5751d9b640 1 -- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c132db0 msgr2=0x7f575c1351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.367+0000 7f5751d9b640 1 --2- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c132db0 0x7f575c1351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.367+0000 7f5751d9b640 1 -- 172.21.15.177:0/2965880272 >> v1:172.21.15.177:6789/0 conn(0x7f575c12d550 legacy=0x7f575c12da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.367+0000 7f5751d9b640 1 -- 172.21.15.177:0/2965880272 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c094b70 con 0x7f575c12e170 2024-09-13T18:36:21.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.367+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5812e150 con 0x7fdb58095100 2024-09-13T18:36:21.372 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.367+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7fdb48004b80 con 0x7fdb58095100 2024-09-13T18:36:21.372 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.368+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7fdb480628f0 con 0x7fdb58095100 2024-09-13T18:36:21.372 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.368+0000 7fdb56ffd640 1 -- 172.21.15.177:0/3196708440 >> v1:172.21.15.177:6816/32265 conn(0x7fdb4404cfc0 legacy=0x7fdb4404f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.368+0000 7f5751d9b640 1 --2- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c12e170 0x7f575c132870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f574c009870 tx=0x7f574c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fbc84600446bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.369+0000 7f5759d9b640 1 -- 172.21.15.177:0/2965880272 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c02ae90 con 0x7f575c12e170 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.369+0000 7f5759d9b640 1 -- 172.21.15.177:0/2965880272 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c032440 con 0x7f575c12e170 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.369+0000 7f5759d9b640 1 -- 172.21.15.177:0/2965880272 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0327d0 con 0x7f575c12e170 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 -- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c12e170 msgr2=0x7f575c132870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 --2- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c12e170 0x7f575c132870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f574c009870 tx=0x7f574c02a180 comp rx=0 tx=0).stop 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 -- 172.21.15.177:0/2965880272 shutdown_connections 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 --2- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c12e170 0x7f575c132870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 --2- 172.21.15.177:0/2965880272 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c132db0 0x7f575c1351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 -- 172.21.15.177:0/2965880272 >> 172.21.15.177:0/2965880272 conn(0x7f575c09ce70 msgr2=0x7f575c09d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 -- 172.21.15.177:0/2965880272 shutdown_connections 2024-09-13T18:36:21.373 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.370+0000 7f5760e12640 1 -- 172.21.15.177:0/2965880272 wait complete. 2024-09-13T18:36:21.374 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 Processor -- start 2024-09-13T18:36:21.374 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 -- start start 2024-09-13T18:36:21.374 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f575c12d550 0x7f575c06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.374 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 0x7f575c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.374 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c06d230 0x7f575c1cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.375 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c06e600 con 0x7f575c132db0 2024-09-13T18:36:21.375 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575c06dec0 con 0x7f575c12d550 2024-09-13T18:36:21.375 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5760e12640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f575c12eab0 con 0x7f575c06d230 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5751d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 0x7f575c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5751d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 0x7f575c06ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47280/0 (socket says 172.21.15.177:47280) 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f5751d9b640 1 -- 172.21.15.177:0/1012775214 learned_addr learned my addr 172.21.15.177:0/1012775214 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f575ad9d640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c06d230 0x7f575c1cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f5751d9b640 1 -- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c06d230 msgr2=0x7f575c1cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f5751d9b640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c06d230 0x7f575c1cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f5751d9b640 1 -- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f575c12d550 msgr2=0x7f575c06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.372+0000 7f575a59c640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f575c12d550 0x7f575c06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f5751d9b640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f575c12d550 0x7f575c06c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f5751d9b640 1 -- 172.21.15.177:0/1012775214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0094d0 con 0x7f575c132db0 2024-09-13T18:36:21.376 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f575a59c640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f575c12d550 0x7f575c06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:21.377 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.373+0000 7f5751d9b640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 0x7f575c06ccf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f574c000c00 tx=0x7f574c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.377 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.374+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c030b80 con 0x7f575c132db0 2024-09-13T18:36:21.377 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.374+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c12ee80 con 0x7f575c132db0 2024-09-13T18:36:21.377 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.375+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c06d6b0 con 0x7f575c132db0 2024-09-13T18:36:21.377 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.375+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c030d00 con 0x7f575c132db0 2024-09-13T18:36:21.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.375+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7fdb4803d6b0 con 0x7fdb58095100 2024-09-13T18:36:21.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.376+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c03b640 con 0x7f575c132db0 2024-09-13T18:36:21.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.376+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f574c039040 con 0x7f575c132db0 2024-09-13T18:36:21.378 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.376+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734005250 con 0x7f575c132db0 2024-09-13T18:36:21.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.376+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f574c088cf0 con 0x7f575c132db0 2024-09-13T18:36:21.379 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.376+0000 7f575a59c640 1 -- 172.21.15.177:0/1012775214 >> v1:172.21.15.177:6816/32265 conn(0x7f574004d140 legacy=0x7f574004f600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.385 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.384+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f574c089060 con 0x7f575c132db0 2024-09-13T18:36:21.424 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.421+0000 7eff70123040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:36:21.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.427+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f74c4127260 con 0x7f74c4125480 2024-09-13T18:36:21.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.427+0000 7f74abfff640 1 -- 172.21.15.177:0/416358894 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f74bc01a070 con 0x7f74c4125480 2024-09-13T18:36:21.429 INFO:teuthology.orchestra.run.smithi177.stdout:38654705681 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 >> v1:172.21.15.177:6816/32265 conn(0x7f74b004cfc0 legacy=0x7f74b004f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c4125480 msgr2=0x7f74c4123390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c4125480 0x7f74c4123390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74bc007db0 tx=0x7f74bc00d4a0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 shutdown_connections 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c4125480 0x7f74c4123390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c4094c90 0x7f74c4122e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 --2- 172.21.15.177:0/416358894 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f74c4092510 0x7f74c4122910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 >> 172.21.15.177:0/416358894 conn(0x7f74c409bfd0 msgr2=0x7f74c40980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 shutdown_connections 2024-09-13T18:36:21.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.429+0000 7f74c9dc7640 1 -- 172.21.15.177:0/416358894 wait complete. 2024-09-13T18:36:21.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.3 2024-09-13T18:36:21.461 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.459+0000 7f516140c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:36:21.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.467+0000 7f4b9f7fe640 1 -- 172.21.15.177:0/1740873093 >> v1:172.21.15.177:6816/32265 conn(0x7f4b8c04d090 legacy=0x7f4b8c04f550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.510 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.508+0000 7eff70123040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:36:21.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.523+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4ba0131660 con 0x7f4ba012a770 2024-09-13T18:36:21.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.524+0000 7f4b87fff640 1 -- 172.21.15.177:0/1740873093 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ba0131660 con 0x7f4ba012a770 2024-09-13T18:36:21.526 INFO:teuthology.orchestra.run.smithi177.stdout:38654705680 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 >> v1:172.21.15.177:6816/32265 conn(0x7f4b8c04d090 legacy=0x7f4b8c04f550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba012a770 msgr2=0x7f4ba0128ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba012a770 0x7f4ba0128ab0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800ca70 tx=0x7f4b9800cf30 comp rx=0 tx=0).stop 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 shutdown_connections 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ba012a770 0x7f4ba0128ab0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ba006ceb0 0x7f4ba0128570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.528 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 --2- 172.21.15.177:0/1740873093 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4ba006c590 0x7f4ba0128030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.529 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 >> 172.21.15.177:0/1740873093 conn(0x7f4ba009ccd0 msgr2=0x7f4ba006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.529 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 shutdown_connections 2024-09-13T18:36:21.529 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.526+0000 7f4ba5b18640 1 -- 172.21.15.177:0/1740873093 wait complete. 2024-09-13T18:36:21.536 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.534+0000 7f516140c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:36:21.537 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.536+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdb58073cc0 con 0x7fdb58095100 2024-09-13T18:36:21.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.536+0000 7fdb3ffff640 1 -- 172.21.15.177:0/3196708440 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb48018020 con 0x7fdb58095100 2024-09-13T18:36:21.538 INFO:teuthology.orchestra.run.smithi177.stdout:38654705680 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 >> v1:172.21.15.177:6816/32265 conn(0x7fdb4404cfc0 legacy=0x7fdb4404f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 msgr2=0x7fdb5812ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 0x7fdb5812ba00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c4d0 tx=0x7fdb4800c990 comp rx=0 tx=0).stop 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 shutdown_connections 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb58137470 0x7fdb5812c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdb58095bf0 0x7fdb5812bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 --2- 172.21.15.177:0/3196708440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58095100 0x7fdb5812ba00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 >> 172.21.15.177:0/3196708440 conn(0x7fdb5809ab20 msgr2=0x7fdb581319c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 shutdown_connections 2024-09-13T18:36:21.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.538+0000 7fdb5dbd4640 1 -- 172.21.15.177:0/3196708440 wait complete. 2024-09-13T18:36:21.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705680 for osd.1 2024-09-13T18:36:21.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705680 for osd.2 2024-09-13T18:36:21.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.577+0000 7f575a59c640 1 -- 172.21.15.177:0/1012775214 >> v1:172.21.15.177:6816/32265 conn(0x7f574004d140 legacy=0x7f574004f600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:21.606 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.604+0000 7eff70123040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:36:21.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.614+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f57340057d0 con 0x7f575c132db0 2024-09-13T18:36:21.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.614+0000 7f57537fe640 1 -- 172.21.15.177:0/1012775214 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f574c042090 con 0x7f575c132db0 2024-09-13T18:36:21.616 INFO:teuthology.orchestra.run.smithi177.stdout:38654705681 2024-09-13T18:36:21.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 >> v1:172.21.15.177:6816/32265 conn(0x7f574004d140 legacy=0x7f574004f600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 msgr2=0x7f575c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 0x7f575c06ccf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f574c000c00 tx=0x7f574c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 shutdown_connections 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f575c06d230 0x7f575c1cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f575c132db0 0x7f575c06ccf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 --2- 172.21.15.177:0/1012775214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f575c12d550 0x7f575c06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 >> 172.21.15.177:0/1012775214 conn(0x7f575c09ce70 msgr2=0x7f575c094e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 shutdown_connections 2024-09-13T18:36:21.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:21.617+0000 7f5760e12640 1 -- 172.21.15.177:0/1012775214 wait complete. 2024-09-13T18:36:21.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.0 2024-09-13T18:36:21.639 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.637+0000 7f516140c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:36:21.703 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.701+0000 7eff70123040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:36:21.735 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.733+0000 7f516140c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:36:21.751 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.749+0000 7eff70123040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:36:21.779 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.777+0000 7f516140c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:36:21.854 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.852+0000 7eff70123040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:36:21.878 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.876+0000 7f516140c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:36:21.909 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.906+0000 7eff70123040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:36:21.930 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.928+0000 7f516140c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:36:21.955 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:21.953+0000 7eff70123040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:36:21.972 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:21.970+0000 7f516140c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:36:22.068 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.066+0000 7eff70123040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:36:22.072 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.071+0000 7f516140c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:36:22.120 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.118+0000 7eff70123040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:36:22.121 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.119+0000 7f516140c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:36:22.175 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.173+0000 7eff70123040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:36:22.175 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.173+0000 7f516140c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:36:22.245 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.242+0000 7f516140c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:36:22.245 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.242+0000 7eff70123040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:36:22.291 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.288+0000 7f516140c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:36:22.291 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.288+0000 7eff70123040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:36:22.336 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.333+0000 7f516140c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:36:22.337 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.333+0000 7eff70123040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:36:22.443 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:22.544 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:22.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.541+0000 7f64dbf76640 1 Processor -- start 2024-09-13T18:36:22.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.541+0000 7f64dbf76640 1 -- start start 2024-09-13T18:36:22.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64dbf76640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d406b440 0x7f64d406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64dbf76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d412dbf0 0x7f64d412e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64dbf76640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d406c4a0 con 0x7f64d406bd60 2024-09-13T18:36:22.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64dbf76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d406d900 con 0x7f64d412dbf0 2024-09-13T18:36:22.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64dbf76640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64d406db60 con 0x7f64d406b440 2024-09-13T18:36:22.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9d12640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d406b440 0x7f64d406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9511640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f64d406bd60 0x7f64d412d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58152/0 (socket says 172.21.15.177:58152) 2024-09-13T18:36:22.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9511640 1 -- 172.21.15.177:0/2814615881 learned_addr learned my addr 172.21.15.177:0/2814615881 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:22.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64da513640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d412dbf0 0x7f64d412e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d8d10640 1 -- 172.21.15.177:0/2814615881 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198084672 0 0) 0x7f64d406c4a0 con 0x7f64d406bd60 2024-09-13T18:36:22.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d8d10640 1 -- 172.21.15.177:0/2814615881 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c0003520 con 0x7f64d406bd60 2024-09-13T18:36:22.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9d12640 1 -- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d412dbf0 msgr2=0x7f64d412e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:22.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9d12640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d412dbf0 0x7f64d412e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9d12640 1 -- 172.21.15.177:0/2814615881 >> v1:172.21.15.177:6789/0 conn(0x7f64d406bd60 legacy=0x7f64d412d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.542+0000 7f64d9d12640 1 -- 172.21.15.177:0/2814615881 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d406c140 con 0x7f64d406b440 2024-09-13T18:36:22.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.543+0000 7f64da513640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d412dbf0 0x7f64d412e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.544+0000 7f64d9d12640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d406b440 0x7f64d406b820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64d00097d0 tx=0x7f64d002a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf407615651287cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.544+0000 7f64d8d10640 1 -- 172.21.15.177:0/2814615881 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d002ad90 con 0x7f64d406b440 2024-09-13T18:36:22.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.544+0000 7f64d8d10640 1 -- 172.21.15.177:0/2814615881 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d0032440 con 0x7f64d406b440 2024-09-13T18:36:22.553 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.544+0000 7f64d8d10640 1 -- 172.21.15.177:0/2814615881 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d00327b0 con 0x7f64d406b440 2024-09-13T18:36:22.555 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:22.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 -- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d406b440 msgr2=0x7f64d406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d406b440 0x7f64d406b820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64d00097d0 tx=0x7f64d002a040 comp rx=0 tx=0).stop 2024-09-13T18:36:22.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 -- 172.21.15.177:0/2814615881 shutdown_connections 2024-09-13T18:36:22.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d412dbf0 0x7f64d412e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2814615881 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d406b440 0x7f64d406b820 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 -- 172.21.15.177:0/2814615881 >> 172.21.15.177:0/2814615881 conn(0x7f64d409fe10 msgr2=0x7f64d40a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 -- 172.21.15.177:0/2814615881 shutdown_connections 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.545+0000 7f64dbf76640 1 -- 172.21.15.177:0/2814615881 wait complete. 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 Processor -- start 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 -- start start 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d406b440 0x7f64d412e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 0x7f64d412eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d412dbf0 0x7f64d412f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4130ab0 con 0x7f64d406bd60 2024-09-13T18:36:22.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4131140 con 0x7f64d406b440 2024-09-13T18:36:22.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64dbf76640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4131510 con 0x7f64d412dbf0 2024-09-13T18:36:22.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64d9511640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 0x7f64d412eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64d9511640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 0x7f64d412eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47292/0 (socket says 172.21.15.177:47292) 2024-09-13T18:36:22.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64d9511640 1 -- 172.21.15.177:0/2805910202 learned_addr learned my addr 172.21.15.177:0/2805910202 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.546+0000 7f64da513640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d412dbf0 0x7f64d412f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9511640 1 -- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d412dbf0 msgr2=0x7f64d412f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9511640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d412dbf0 0x7f64d412f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9511640 1 -- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d406b440 msgr2=0x7f64d412e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9d12640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d406b440 0x7f64d412e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.563 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9511640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d406b440 0x7f64d412e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.563 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9511640 1 -- 172.21.15.177:0/2805910202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc009a70 con 0x7f64d406bd60 2024-09-13T18:36:22.563 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9d12640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d406b440 0x7f64d412e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:22.563 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64d9511640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 0x7f64d412eb10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c500 tx=0x7f64c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.563 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4005ce0 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4005e40 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4007330 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d0009400 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.547+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d412f810 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.548+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d4130880 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.549+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f64c4007490 con 0x7f64d406bd60 2024-09-13T18:36:22.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.549+0000 7f64d9d12640 1 -- 172.21.15.177:0/2805910202 >> v1:172.21.15.177:6816/32265 conn(0x7f64b804cfc0 legacy=0x7f64b804f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.177:6816/32265 2024-09-13T18:36:22.565 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.549+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f64c4062950 con 0x7f64d406bd60 2024-09-13T18:36:22.565 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.553+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f64c403d810 con 0x7f64d406bd60 2024-09-13T18:36:22.574 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-09-13T18:36:22.572+0000 7f516140c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:36:22.575 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:22.572+0000 7eff70123040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:36:22.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.584+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mgrmap(e 7) v1 ==== 54958+0+0 (secure 0 0 0) 0x7f64c4061600 con 0x7f64d406bd60 2024-09-13T18:36:22.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.584+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 >> v1:172.21.15.177:6816/32265 conn(0x7f64b804cfc0 legacy=0x7f64b804f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:22.636 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:22.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.659+0000 7f832b9e2640 1 Processor -- start 2024-09-13T18:36:22.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.659+0000 7f832b9e2640 1 -- start start 2024-09-13T18:36:22.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832b9e2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f832406b480 0x7f832406b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832b9e2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f832406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832b9e2640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406c2e0 0x7f8324133c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832b9e2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832406d890 con 0x7f832406b480 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f832406b480 0x7f832406b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832b9e2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8324066bb0 con 0x7f832406d190 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f8328f7d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f832406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f832406b480 0x7f832406b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47358/0 (socket says 172.21.15.177:47358) 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f8329f7f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406c2e0 0x7f8324133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f8328f7d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f832406bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35206/0 (socket says 172.21.15.177:35206) 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 -- 172.21.15.177:0/4276993657 learned_addr learned my addr 172.21.15.177:0/4276993657 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832b9e2640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8324071380 con 0x7f832406c2e0 2024-09-13T18:36:22.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 -- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406c2e0 msgr2=0x7f8324133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:22.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406c2e0 0x7f8324133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 -- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 msgr2=0x7f832406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:22.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f832406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.660+0000 7f832977e640 1 -- 172.21.15.177:0/4276993657 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324134170 con 0x7f832406b480 2024-09-13T18:36:22.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.661+0000 7f8328f7d640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f832406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.661+0000 7f8329f7f640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406c2e0 0x7f8324133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.662+0000 7f832977e640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406b480 0x7f832406b860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8320009870 tx=0x7f832002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873f6166872bf167 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.662+0000 7f8317fff640 1 -- 172.21.15.177:0/4276993657 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832002adf0 con 0x7f832406b480 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.662+0000 7f8317fff640 1 -- 172.21.15.177:0/4276993657 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8320032440 con 0x7f832406b480 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.662+0000 7f8317fff640 1 -- 172.21.15.177:0/4276993657 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200327d0 con 0x7f832406b480 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 -- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406b480 msgr2=0x7f832406b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406b480 0x7f832406b860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8320009870 tx=0x7f832002a0a0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 -- 172.21.15.177:0/4276993657 shutdown_connections 2024-09-13T18:36:22.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406c2e0 0x7f8324133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f832406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 --2- 172.21.15.177:0/4276993657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406b480 0x7f832406b860 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 -- 172.21.15.177:0/4276993657 >> 172.21.15.177:0/4276993657 conn(0x7f832409ca80 msgr2=0x7f832409ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 -- 172.21.15.177:0/4276993657 shutdown_connections 2024-09-13T18:36:22.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.663+0000 7f832b9e2640 1 -- 172.21.15.177:0/4276993657 wait complete. 2024-09-13T18:36:22.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.664+0000 7f832b9e2640 1 Processor -- start 2024-09-13T18:36:22.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.664+0000 7f832b9e2640 1 -- start start 2024-09-13T18:36:22.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832b9e2640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 0x7f83241227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832b9e2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 0x7f8324122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832b9e2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f8324123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832977e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 0x7f83241227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f8328f7d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 0x7f8324122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832977e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 0x7f83241227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32886/0 (socket says 172.21.15.177:32886) 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f8328f7d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 0x7f8324122d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47374/0 (socket says 172.21.15.177:47374) 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832977e640 1 -- 172.21.15.177:0/2054674703 learned_addr learned my addr 172.21.15.177:0/2054674703 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832b9e2640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8324124260 con 0x7f832406c2e0 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832b9e2640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8324096450 con 0x7f832406d190 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832b9e2640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8324124f40 con 0x7f832406b480 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f8329f7f640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f8324123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.665+0000 7f832977e640 1 -- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 msgr2=0x7f8324123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832977e640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f8324123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832977e640 1 -- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 msgr2=0x7f8324122d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832977e640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 0x7f8324122d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832977e640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c009a70 con 0x7f832406b480 2024-09-13T18:36:22.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f8328f7d640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 0x7f8324122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f8329f7f640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f8324123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832977e640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 0x7f83241227e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f832002a5c0 tx=0x7f83200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832003d830 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832b9e2640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8318009400 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832003d990 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832003d830 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.666+0000 7f832b9e2640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83200094d0 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.668+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 7) v1 ==== 54958+0+0 (secure 0 0 0) 0x7f8320039040 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.668+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc005250 con 0x7f832406b480 2024-09-13T18:36:22.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.668+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f832007cbd0 con 0x7f832406b480 2024-09-13T18:36:22.679 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.677+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f8320047bb0 con 0x7f832406b480 2024-09-13T18:36:22.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.679+0000 7f16dae2b640 1 Processor -- start 2024-09-13T18:36:22.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.679+0000 7f16dae2b640 1 -- start start 2024-09-13T18:36:22.685 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16dae2b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.685 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16dae2b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.685 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16dae2b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f16d412da20 0x7f16d412dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16dae2b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d406d840 con 0x7f16d412da20 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16dae2b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d406c4a0 con 0x7f16d406b440 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16dae2b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16d406db60 con 0x7f16d406bd60 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d406b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35238/0 (socket says 172.21.15.177:35238) 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 -- 172.21.15.177:0/3624949133 learned_addr learned my addr 172.21.15.177:0/3624949133 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d3fff640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d93c8640 1 --2- 172.21.15.177:0/3624949133 >> v2:172.21.15.177:3300/0 conn(0x7f16d412da20 0x7f16d412dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 -- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 msgr2=0x7f16d412d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 -- 172.21.15.177:0/3624949133 >> v2:172.21.15.177:3300/0 conn(0x7f16d412da20 msgr2=0x7f16d412dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:22.686 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 --2- 172.21.15.177:0/3624949133 >> v2:172.21.15.177:3300/0 conn(0x7f16d412da20 0x7f16d412dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d8bc7640 1 -- 172.21.15.177:0/3624949133 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d406c140 con 0x7f16d406b440 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d93c8640 1 --2- 172.21.15.177:0/3624949133 >> v2:172.21.15.177:3300/0 conn(0x7f16d412da20 0x7f16d412dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.680+0000 7f16d3fff640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.681+0000 7f16d8bc7640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d406b820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009870 tx=0x7f16c802a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1cc3d89888a7d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.681+0000 7f16d37fe640 1 -- 172.21.15.177:0/3624949133 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c802ade0 con 0x7f16d406b440 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.681+0000 7f16d37fe640 1 -- 172.21.15.177:0/3624949133 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8032440 con 0x7f16d406b440 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.681+0000 7f16d37fe640 1 -- 172.21.15.177:0/3624949133 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80327d0 con 0x7f16d406b440 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 -- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 msgr2=0x7f16d406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d406b820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009870 tx=0x7f16c802a090 comp rx=0 tx=0).stop 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 -- 172.21.15.177:0/3624949133 shutdown_connections 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 --2- 172.21.15.177:0/3624949133 >> v2:172.21.15.177:3300/0 conn(0x7f16d412da20 0x7f16d412dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 --2- 172.21.15.177:0/3624949133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d406b820 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 -- 172.21.15.177:0/3624949133 >> 172.21.15.177:0/3624949133 conn(0x7f16d409fe10 msgr2=0x7f16d40a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 -- 172.21.15.177:0/3624949133 shutdown_connections 2024-09-13T18:36:22.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.682+0000 7f16dae2b640 1 -- 172.21.15.177:0/3624949133 wait complete. 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.683+0000 7f16dae2b640 1 Processor -- start 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.683+0000 7f16dae2b640 1 -- start start 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16dae2b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d412e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16dae2b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16dae2b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d412da20 0x7f16d412f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16d3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.690 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16d3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32894/0 (socket says 172.21.15.177:32894) 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16d3fff640 1 -- 172.21.15.177:0/150024740 learned_addr learned my addr 172.21.15.177:0/150024740 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16d8bc7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d412e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16d93c8640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d412da20 0x7f16d412f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16dae2b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d412fb80 con 0x7f16d412da20 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d4127f00 con 0x7f16d406b440 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.684+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16d412f800 con 0x7f16d406bd60 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d3fff640 1 -- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 msgr2=0x7f16d412e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d3fff640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d412e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d3fff640 1 -- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d412da20 msgr2=0x7f16d412f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d3fff640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d412da20 0x7f16d412f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d3fff640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009400 con 0x7f16d406bd60 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d93c8640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d412da20 0x7f16d412f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d3fff640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412ed80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16c000ebe0 tx=0x7f16c000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.691 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c000eee0 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c000cc30 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc009a70 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0009760 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.685+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c80094d0 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.686+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d40a0750 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.687+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 7) v1 ==== 54958+0+0 (secure 0 0 0) 0x7f16c0013470 con 0x7f16d406bd60 2024-09-13T18:36:22.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.687+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f16c005c730 con 0x7f16d406bd60 2024-09-13T18:36:22.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.690+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f16c003a6c0 con 0x7f16d406bd60 2024-09-13T18:36:22.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.731+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64d406b820 con 0x7f64d406bd60 2024-09-13T18:36:22.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.731+0000 7f64ca7fc640 1 -- 172.21.15.177:0/2805910202 <== mon.0 v2:172.21.15.177:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64c4017090 con 0x7f64d406bd60 2024-09-13T18:36:22.735 INFO:teuthology.orchestra.run.smithi177.stdout:38654705681 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 msgr2=0x7f64d412eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 0x7f64d412eb10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c500 tx=0x7f64c400c9c0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 shutdown_connections 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f64d412dbf0 0x7f64d412f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d406bd60 0x7f64d412eb10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 --2- 172.21.15.177:0/2805910202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d406b440 0x7f64d412e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 >> 172.21.15.177:0/2805910202 conn(0x7f64d409fe10 msgr2=0x7f64d41273b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 shutdown_connections 2024-09-13T18:36:22.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.733+0000 7f64dbf76640 1 -- 172.21.15.177:0/2805910202 wait complete. 2024-09-13T18:36:22.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.736+0000 7fe8d95a0640 1 Processor -- start 2024-09-13T18:36:22.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d95a0640 1 -- start start 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d95a0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 0x7fe8d412f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d95a0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe8d412fb80 0x7fe8d4131f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d95a0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d406dbc0 con 0x7fe8d406c500 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d95a0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d406ece0 con 0x7fe8d412fb80 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d95a0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d406d820 con 0x7fe8d412d250 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 0x7fe8d412f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d8d9f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe8d412fb80 0x7fe8d4131f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 0x7fe8d412f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32898/0 (socket says 172.21.15.177:32898) 2024-09-13T18:36:22.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d8d9f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe8d412fb80 0x7fe8d4131f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47396/0 (socket says 172.21.15.177:47396) 2024-09-13T18:36:22.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d37fe640 1 -- 172.21.15.177:0/3870123798 learned_addr learned my addr 172.21.15.177:0/3870123798 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.737+0000 7fe8d3fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe8d406c500 0x7fe8d406c8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58156/0 (socket says 172.21.15.177:58156) 2024-09-13T18:36:22.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.738+0000 7fe8d37fe640 1 -- 172.21.15.177:0/3870123798 >> v2:172.21.15.177:3300/0 conn(0x7fe8d412fb80 msgr2=0x7fe8d4131f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:22.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.738+0000 7fe8d37fe640 1 --2- 172.21.15.177:0/3870123798 >> v2:172.21.15.177:3300/0 conn(0x7fe8d412fb80 0x7fe8d4131f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.738+0000 7fe8d37fe640 1 -- 172.21.15.177:0/3870123798 >> v1:172.21.15.177:6789/0 conn(0x7fe8d406c500 legacy=0x7fe8d406c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.738+0000 7fe8d37fe640 1 -- 172.21.15.177:0/3870123798 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d406d3d0 con 0x7fe8d412d250 2024-09-13T18:36:22.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.739+0000 7fe8d37fe640 1 --2- 172.21.15.177:0/3870123798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 0x7fe8d412f640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800bb10 tx=0x7fe8c802c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5703e16d237b505c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.739+0000 7fe8d2ffd640 1 -- 172.21.15.177:0/3870123798 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8031710 con 0x7fe8d412d250 2024-09-13T18:36:22.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.739+0000 7fe8d2ffd640 1 -- 172.21.15.177:0/3870123798 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c802f050 con 0x7fe8d412d250 2024-09-13T18:36:22.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.739+0000 7fe8d2ffd640 1 -- 172.21.15.177:0/3870123798 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8031a90 con 0x7fe8d412d250 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 -- 172.21.15.177:0/3870123798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 msgr2=0x7fe8d412f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/3870123798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 0x7fe8d412f640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800bb10 tx=0x7fe8c802c700 comp rx=0 tx=0).stop 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 -- 172.21.15.177:0/3870123798 shutdown_connections 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/3870123798 >> v2:172.21.15.177:3300/0 conn(0x7fe8d412fb80 0x7fe8d4131f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/3870123798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d412d250 0x7fe8d412f640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 -- 172.21.15.177:0/3870123798 >> 172.21.15.177:0/3870123798 conn(0x7fe8d409cb10 msgr2=0x7fe8d409cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 -- 172.21.15.177:0/3870123798 shutdown_connections 2024-09-13T18:36:22.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.740+0000 7fe8d95a0640 1 -- 172.21.15.177:0/3870123798 wait complete. 2024-09-13T18:36:22.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 Processor -- start 2024-09-13T18:36:22.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 -- start start 2024-09-13T18:36:22.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 0x7fe8d408af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8d412d250 0x7fe8d408b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8d412fb80 0x7fe8d408b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:22.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d408d800 con 0x7fe8d412fb80 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d408da50 con 0x7fe8d412d250 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.741+0000 7fe8d95a0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d408cea0 con 0x7fe8d406c500 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 0x7fe8d408af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 0x7fe8d408af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32908/0 (socket says 172.21.15.177:32908) 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 -- 172.21.15.177:0/1645836781 learned_addr learned my addr 172.21.15.177:0/1645836781 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d8d9f640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8d412fb80 0x7fe8d408b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d37fe640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8d412d250 0x7fe8d408b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 -- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8d412d250 msgr2=0x7fe8d408b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8d412d250 0x7fe8d408b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.749 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 -- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8d412fb80 msgr2=0x7fe8d408b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8d412fb80 0x7fe8d408b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 -- 172.21.15.177:0/1645836781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c800b770 con 0x7fe8d406c500 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d37fe640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8d412d250 0x7fe8d408b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d3fff640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 0x7fe8d408af60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c780 tx=0x7fe8c400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40074e0 con 0x7fe8d406c500 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4007640 con 0x7fe8d406c500 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d408c5f0 con 0x7fe8d406c500 2024-09-13T18:36:22.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4004a00 con 0x7fe8d406c500 2024-09-13T18:36:22.751 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.742+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d4126fd0 con 0x7fe8d406c500 2024-09-13T18:36:22.751 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.744+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4070bd0 con 0x7fe8d406c500 2024-09-13T18:36:22.751 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.744+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 7) v1 ==== 54958+0+0 (secure 0 0 0) 0x7fe8c4004b60 con 0x7fe8d406c500 2024-09-13T18:36:22.751 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.744+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fe8c4063300 con 0x7fe8d406c500 2024-09-13T18:36:22.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.3 2024-09-13T18:36:22.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.748+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7fe8c4061060 con 0x7fe8d406c500 2024-09-13T18:36:22.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.831+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f16d412d4e0 con 0x7f16d406bd60 2024-09-13T18:36:22.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.831+0000 7f16d17fa640 1 -- 172.21.15.177:0/150024740 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16c003dce0 con 0x7f16d406bd60 2024-09-13T18:36:22.835 INFO:teuthology.orchestra.run.smithi177.stdout:38654705680 2024-09-13T18:36:22.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.832+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f82fc005500 con 0x7f832406b480 2024-09-13T18:36:22.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.832+0000 7f8315ffb640 1 -- 172.21.15.177:0/2054674703 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8320037020 con 0x7f832406b480 2024-09-13T18:36:22.836 INFO:teuthology.orchestra.run.smithi177.stdout:38654705680 2024-09-13T18:36:22.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 msgr2=0x7f16d412ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412ed80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16c000ebe0 tx=0x7f16c000c520 comp rx=0 tx=0).stop 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 shutdown_connections 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d412da20 0x7f16d412f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f16d406bd60 0x7f16d412ed80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 --2- 172.21.15.177:0/150024740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d406b440 0x7f16d412e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 >> 172.21.15.177:0/150024740 conn(0x7f16d409fe10 msgr2=0x7f16d409bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 shutdown_connections 2024-09-13T18:36:22.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.833+0000 7f16dae2b640 1 -- 172.21.15.177:0/150024740 wait complete. 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 msgr2=0x7f83241227e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 0x7f83241227e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f832002a5c0 tx=0x7f83200046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 shutdown_connections 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832406d190 0x7f8324123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832406c2e0 0x7f8324122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 --2- 172.21.15.177:0/2054674703 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f832406b480 0x7f83241227e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 >> 172.21.15.177:0/2054674703 conn(0x7f832409ca80 msgr2=0x7f832406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 shutdown_connections 2024-09-13T18:36:22.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.834+0000 7f830f7fe640 1 -- 172.21.15.177:0/2054674703 wait complete. 2024-09-13T18:36:22.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705680 for osd.2 2024-09-13T18:36:22.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705680 for osd.1 2024-09-13T18:36:22.889 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.886+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe8d408c780 con 0x7fe8d406c500 2024-09-13T18:36:22.889 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.886+0000 7fe8d0ff9640 1 -- 172.21.15.177:0/1645836781 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe8c4018020 con 0x7fe8d406c500 2024-09-13T18:36:22.890 INFO:teuthology.orchestra.run.smithi177.stdout:38654705681 2024-09-13T18:36:22.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 msgr2=0x7fe8d408af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:22.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 0x7fe8d408af60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c780 tx=0x7fe8c400cc40 comp rx=0 tx=0).stop 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 shutdown_connections 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8d412fb80 0x7fe8d408b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8d412d250 0x7fe8d408b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 --2- 172.21.15.177:0/1645836781 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe8d406c500 0x7fe8d408af60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 >> 172.21.15.177:0/1645836781 conn(0x7fe8d409cb10 msgr2=0x7fe8d412afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 shutdown_connections 2024-09-13T18:36:22.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:22.888+0000 7fe8d95a0640 1 -- 172.21.15.177:0/1645836781 wait complete. 2024-09-13T18:36:22.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.0 2024-09-13T18:36:23.589 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:23.586+0000 7eff59995640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-13T18:36:23.590 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:23.586+0000 7eff59995640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T18:36:23.591 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-13T18:36:23.589+0000 7eff59995640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T18:36:23.753 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:23.853 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:23.853 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:23.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 Processor -- start 2024-09-13T18:36:23.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 -- start start 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc4012b690 0x7fdc4012ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdc40095810 0x7fdc401335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4012b280 con 0x7fdc40099db0 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc40096a20 con 0x7fdc40095810 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc45e2b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40096cf0 con 0x7fdc4012b690 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc3ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdc40095810 0x7fdc401335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc4012b690 0x7fdc4012ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc3ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdc40095810 0x7fdc401335e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47408/0 (socket says 172.21.15.177:47408) 2024-09-13T18:36:23.858 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.854+0000 7fdc3ffff640 1 -- 172.21.15.177:0/316522203 learned_addr learned my addr 172.21.15.177:0/316522203 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:23.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.855+0000 7fdc3ffff640 1 -- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc4012b690 msgr2=0x7fdc4012ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.855+0000 7fdc3ffff640 1 --2- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc4012b690 0x7fdc4012ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.855+0000 7fdc3ffff640 1 -- 172.21.15.177:0/316522203 >> v1:172.21.15.177:6789/0 conn(0x7fdc40099db0 legacy=0x7fdc40095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.855+0000 7fdc3effd640 1 -- 172.21.15.177:0/316522203 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912315046 0 0) 0x7fdc4012b280 con 0x7fdc40099db0 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.855+0000 7fdc3ffff640 1 -- 172.21.15.177:0/316522203 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400961b0 con 0x7fdc40095810 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.855+0000 7fdc3f7fe640 1 --2- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc4012b690 0x7fdc4012ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc3ffff640 1 --2- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc40095810 0x7fdc401335e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009e50 tx=0x7fdc3802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee5cc2aa5ea8af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc3effd640 1 -- 172.21.15.177:0/316522203 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38031600 con 0x7fdc40095810 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc3effd640 1 -- 172.21.15.177:0/316522203 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3802d050 con 0x7fdc40095810 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc3effd640 1 -- 172.21.15.177:0/316522203 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38031980 con 0x7fdc40095810 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc45e2b640 1 -- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc40095810 msgr2=0x7fdc401335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc40095810 0x7fdc401335e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009e50 tx=0x7fdc3802a610 comp rx=0 tx=0).stop 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.856+0000 7fdc45e2b640 1 -- 172.21.15.177:0/316522203 shutdown_connections 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc40095810 0x7fdc401335e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/316522203 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc4012b690 0x7fdc4012ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 -- 172.21.15.177:0/316522203 >> 172.21.15.177:0/316522203 conn(0x7fdc4009ab20 msgr2=0x7fdc4009af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:23.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 -- 172.21.15.177:0/316522203 shutdown_connections 2024-09-13T18:36:23.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 -- 172.21.15.177:0/316522203 wait complete. 2024-09-13T18:36:23.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 Processor -- start 2024-09-13T18:36:23.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.857+0000 7fdc45e2b640 1 -- start start 2024-09-13T18:36:23.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc45e2b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 0x7fdc4008c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc45e2b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc40099db0 0x7fdc4008c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc45e2b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc4012b690 0x7fdc401c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc45e2b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4008f280 con 0x7fdc4012b690 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc45e2b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4008d3f0 con 0x7fdc40099db0 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc45e2b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4008e130 con 0x7fdc40095810 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 0x7fdc4008c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 0x7fdc4008c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32926/0 (socket says 172.21.15.177:32926) 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 -- 172.21.15.177:0/766281274 learned_addr learned my addr 172.21.15.177:0/766281274 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3ffff640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc4012b690 0x7fdc401c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc37fff640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc40099db0 0x7fdc4008c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 -- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc40099db0 msgr2=0x7fdc4008c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc40099db0 0x7fdc4008c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 -- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc4012b690 msgr2=0x7fdc401c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc4012b690 0x7fdc401c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 -- 172.21.15.177:0/766281274 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30009400 con 0x7fdc40095810 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.858+0000 7fdc3f7fe640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 0x7fdc4008c460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000ebe0 tx=0x7fdc3000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.859+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30009760 con 0x7fdc40095810 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.859+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3000cc30 con 0x7fdc40095810 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.859+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000eee0 con 0x7fdc40095810 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.859+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38009b30 con 0x7fdc40095810 2024-09-13T18:36:23.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.859+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4008dd90 con 0x7fdc40095810 2024-09-13T18:36:23.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.860+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7fdc30013470 con 0x7fdc40095810 2024-09-13T18:36:23.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.860+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4012ba70 con 0x7fdc40095810 2024-09-13T18:36:23.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.861+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fdc3005c9c0 con 0x7fdc40095810 2024-09-13T18:36:23.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.865+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fdc3003a750 con 0x7fdc40095810 2024-09-13T18:36:23.903 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 Processor -- start 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 -- start start 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7f868a108640 1 Processor -- start 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7f868a108640 1 -- start start 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc90006b440 0x7fc90006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90006bd60 0x7fc90012d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc90012deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90006d840 con 0x7fc90006b440 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90006c4a0 con 0x7fc90006bd60 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.953+0000 7fc9049f4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc90006db60 con 0x7fc90012da20 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fdd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90006bd60 0x7fc90012d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fdd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90006bd60 0x7fc90012d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35284/0 (socket says 172.21.15.177:35284) 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fdd9b640 1 -- 172.21.15.177:0/1237845030 learned_addr learned my addr 172.21.15.177:0/1237845030 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fe59c640 1 --2- 172.21.15.177:0/1237845030 >> v2:172.21.15.177:3300/0 conn(0x7fc90006b440 0x7fc90006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f868a108640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f868412d5a0 0x7f868412f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f868a108640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f868412fed0 0x7f86841322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f868a108640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8684126dd0 con 0x7f8684127080 2024-09-13T18:36:23.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f868a108640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8684127b70 con 0x7f868412d5a0 2024-09-13T18:36:23.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f868a108640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8684127ce0 con 0x7f868412fed0 2024-09-13T18:36:23.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f86837fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f8684127080 0x7f8684127460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58168/0 (socket says 172.21.15.177:58168) 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f86837fe640 1 -- 172.21.15.177:0/4084742555 learned_addr learned my addr 172.21.15.177:0/4084742555 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fed9d640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc90012deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f8683fff640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f868412fed0 0x7f86841322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f8682ffd640 1 --2- 172.21.15.177:0/4084742555 >> v2:172.21.15.177:3300/0 conn(0x7f868412d5a0 0x7f868412f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318726200 0 0) 0x7f8684126dd0 con 0x7f8684127080 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fe59c640 1 -- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 msgr2=0x7fc90012deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c003520 con 0x7f8684127080 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 <== mon.0 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3296201331 0 0) 0x7f866c003520 con 0x7f8684127080 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f868412fed0 msgr2=0x7f86841322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f868412fed0 0x7f86841322c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 >> v2:172.21.15.177:3300/0 conn(0x7f868412d5a0 msgr2=0x7f868412f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 --2- 172.21.15.177:0/4084742555 >> v2:172.21.15.177:3300/0 conn(0x7f868412d5a0 0x7f868412f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684070e00 con 0x7f8684127080 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 <== mon.0 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2334991528 0 0) 0x7f8674001670 con 0x7f8684127080 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 >> v1:172.21.15.177:6789/0 conn(0x7f8684127080 legacy=0x7f8684127460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f866c003970 0x7f866c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f866c0047c0 0x7f866c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f866c005180 0x7f866c014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684127b70 con 0x7f866c005180 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8684127ce0 con 0x7f866c003970 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f866c014780 con 0x7f866c0047c0 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f86837fe640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f866c003970 0x7f866c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fe59c640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc90012deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fe59c640 1 -- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90006bd60 msgr2=0x7fc90012d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fe59c640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90006bd60 0x7fc90012d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fe59c640 1 -- 172.21.15.177:0/1237845030 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90006c170 con 0x7fc90006b440 2024-09-13T18:36:23.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.954+0000 7fc8fed9d640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc90012deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:23.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f8682ffd640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f866c0047c0 0x7f866c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.955+0000 7f8683fff640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f866c005180 0x7f866c014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7f86837fe640 1 -- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f866c0047c0 msgr2=0x7f866c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7f86837fe640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f866c0047c0 0x7f866c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7f86837fe640 1 -- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f866c005180 msgr2=0x7f866c014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7f86837fe640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f866c005180 0x7f866c014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7f86837fe640 1 -- 172.21.15.177:0/4084742555 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c003520 con 0x7f866c003970 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7fc8fe59c640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006b440 0x7fc90006b820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00097a0 tx=0x7fc8f0029fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb369784ed8ed51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7fc8fd59a640 1 -- 172.21.15.177:0/1237845030 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f002ad10 con 0x7fc90006b440 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7fc8fd59a640 1 -- 172.21.15.177:0/1237845030 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f002ae70 con 0x7fc90006b440 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.956+0000 7fc8fd59a640 1 -- 172.21.15.177:0/1237845030 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0032640 con 0x7fc90006b440 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7f86837fe640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f866c003970 0x7f866c003d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f867400c240 tx=0x7f866c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b82d9aee13795ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674005ad0 con 0x7f866c003970 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674005c30 con 0x7f866c003970 2024-09-13T18:36:23.967 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7f86827fc640 1 -- 172.21.15.177:0/4084742555 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674007070 con 0x7f866c003970 2024-09-13T18:36:23.967 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 -- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006b440 msgr2=0x7fc90006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006b440 0x7fc90006b820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00097a0 tx=0x7fc8f0029fc0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 -- 172.21.15.177:0/1237845030 shutdown_connections 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc90012deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90006bd60 0x7fc90012d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 --2- 172.21.15.177:0/1237845030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006b440 0x7fc90006b820 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 -- 172.21.15.177:0/1237845030 >> 172.21.15.177:0/1237845030 conn(0x7fc90009fe10 msgr2=0x7fc9000a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 -- 172.21.15.177:0/1237845030 shutdown_connections 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7f868a108640 1 -- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f866c003970 msgr2=0x7f866c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7f868a108640 1 --2- 172.21.15.177:0/4084742555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f866c003970 0x7f866c003d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f867400c240 tx=0x7f866c0057f0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.969 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.957+0000 7fc9049f4640 1 -- 172.21.15.177:0/1237845030 wait complete. 2024-09-13T18:36:23.969 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7f8683fff640 1 -- 172.21.15.177:0/4084742555 reap_dead start 2024-09-13T18:36:23.969 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7f868a108640 1 -- 172.21.15.177:0/4084742555 shutdown_connections 2024-09-13T18:36:23.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7f868a108640 1 -- 172.21.15.177:0/4084742555 >> 172.21.15.177:0/4084742555 conn(0x7f868409b680 msgr2=0x7f868409ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:23.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7f868a108640 1 -- 172.21.15.177:0/4084742555 shutdown_connections 2024-09-13T18:36:23.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7f868a108640 1 -- 172.21.15.177:0/4084742555 wait complete. 2024-09-13T18:36:23.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7f868a108640 1 Processor -- start 2024-09-13T18:36:23.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.958+0000 7fc9049f4640 1 Processor -- start 2024-09-13T18:36:23.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 -- start start 2024-09-13T18:36:23.971 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 -- start start 2024-09-13T18:36:23.971 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006bd60 0x7fc900099680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.971 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc900099bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90012e540 0x7fc90012e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc900131130 con 0x7fc90006bd60 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9001319e0 con 0x7fc90012e540 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc9049f4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9001316d0 con 0x7fc90012da20 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc8fdd9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc900099bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc8fed9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90012e540 0x7fc90012e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868411fd90 0x7f8684120170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86841206b0 0x7f8684121830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 0x7f8684122220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc8fdd9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc900099bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32952/0 (socket says 172.21.15.177:32952) 2024-09-13T18:36:23.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc8fdd9b640 1 -- 172.21.15.177:0/3025830290 learned_addr learned my addr 172.21.15.177:0/3025830290 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:23.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684113830 con 0x7f868411fd90 2024-09-13T18:36:23.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8684123a80 con 0x7f86841206b0 2024-09-13T18:36:23.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f868a108640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8684123d30 con 0x7f8684121d70 2024-09-13T18:36:23.974 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f8683fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 0x7f8684122220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.974 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f8683fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 0x7f8684122220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:32968/0 (socket says 172.21.15.177:32968) 2024-09-13T18:36:23.974 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f8683fff640 1 -- 172.21.15.177:0/3383934148 learned_addr learned my addr 172.21.15.177:0/3383934148 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:23.974 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7fc8fe59c640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006bd60 0x7fc900099680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.974 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.959+0000 7f8682ffd640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86841206b0 0x7f8684121830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fed9d640 1 -- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 msgr2=0x7fc900099bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fed9d640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc900099bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fed9d640 1 -- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006bd60 msgr2=0x7fc900099680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f86837fe640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868411fd90 0x7f8684120170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8683fff640 1 -- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86841206b0 msgr2=0x7f8684121830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fed9d640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006bd60 0x7fc900099680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fed9d640 1 -- 172.21.15.177:0/3025830290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8009a70 con 0x7fc90012e540 2024-09-13T18:36:23.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8683fff640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86841206b0 0x7f8684121830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8683fff640 1 -- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868411fd90 msgr2=0x7f8684120170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8683fff640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868411fd90 0x7f8684120170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8683fff640 1 -- 172.21.15.177:0/3383934148 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684123050 con 0x7f8684121d70 2024-09-13T18:36:23.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f86837fe640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868411fd90 0x7f8684120170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:23.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8682ffd640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86841206b0 0x7f8684121830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:23.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fed9d640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90012e540 0x7fc90012e9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009e50 tx=0x7fc8f800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f8683fff640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 0x7f8684122220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f868406a560 tx=0x7f867c0115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fdd9b640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc900099bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:23.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8fe59c640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006bd60 0x7fc900099680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:23.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00e070 con 0x7f8684121d70 2024-09-13T18:36:23.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c011db0 con 0x7f8684121d70 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c003990 con 0x7f8684121d70 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7f868a108640 1 -- 172.21.15.177:0/3383934148 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8684124380 con 0x7f8684121d70 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.961+0000 7f868a108640 1 -- 172.21.15.177:0/3383934148 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8684122940 con 0x7f8684121d70 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.960+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800cd90 con 0x7fc90012e540 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.961+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f800f040 con 0x7fc90012e540 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.961+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8003bd0 con 0x7fc90012e540 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.961+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f0009400 con 0x7fc90012e540 2024-09-13T18:36:23.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.961+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90009a100 con 0x7fc90012e540 2024-09-13T18:36:23.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.962+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8684124100 con 0x7f8684121d70 2024-09-13T18:36:23.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.962+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f867c003af0 con 0x7f8684121d70 2024-09-13T18:36:23.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.962+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f867c05cd90 con 0x7f8684121d70 2024-09-13T18:36:23.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.963+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90012d2a0 con 0x7fc90012e540 2024-09-13T18:36:23.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.963+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7fc8f8003d30 con 0x7fc90012e540 2024-09-13T18:36:23.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.963+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fc8f805d090 con 0x7fc90012e540 2024-09-13T18:36:23.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.966+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f867c03ac50 con 0x7f8684121d70 2024-09-13T18:36:23.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:23.967+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fc8f803ae50 con 0x7fc90012e540 2024-09-13T18:36:24.009 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 Processor -- start 2024-09-13T18:36:24.009 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 -- start start 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f61f8095810 0x7f61f8132da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095c20 0x7f61f81352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f8099f30 con 0x7f61f8094b20 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f809a100 con 0x7f61f8095810 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fe93e640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8096990 con 0x7f61f8095c20 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61f7fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f61f8094b20 0x7f61f8095100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58180/0 (socket says 172.21.15.177:58180) 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61f7fff640 1 -- 172.21.15.177:0/585131621 learned_addr learned my addr 172.21.15.177:0/585131621 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61fcedb640 1 --2- 172.21.15.177:0/585131621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095c20 0x7f61f81352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:24.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.006+0000 7f61f77fe640 1 --2- 172.21.15.177:0/585131621 >> v2:172.21.15.177:3300/0 conn(0x7f61f8095810 0x7f61f8132da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.007+0000 7f61fcedb640 1 -- 172.21.15.177:0/585131621 >> v2:172.21.15.177:3300/0 conn(0x7f61f8095810 msgr2=0x7f61f8132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.007+0000 7f61fcedb640 1 --2- 172.21.15.177:0/585131621 >> v2:172.21.15.177:3300/0 conn(0x7f61f8095810 0x7f61f8132da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.007+0000 7f61fcedb640 1 -- 172.21.15.177:0/585131621 >> v1:172.21.15.177:6789/0 conn(0x7f61f8094b20 legacy=0x7f61f8095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.007+0000 7f61fcedb640 1 -- 172.21.15.177:0/585131621 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f809b640 con 0x7f61f8095c20 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.007+0000 7f61f6ffd640 1 -- 172.21.15.177:0/585131621 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382479410 0 0) 0x7f61f8099f30 con 0x7f61f8094b20 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.008+0000 7f61fcedb640 1 --2- 172.21.15.177:0/585131621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095c20 0x7f61f81352f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009dc0 tx=0x7f61f002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66dab565260ccff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.008+0000 7f61f6ffd640 1 -- 172.21.15.177:0/585131621 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0007740 con 0x7f61f8095c20 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.008+0000 7f61f6ffd640 1 -- 172.21.15.177:0/585131621 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f002d050 con 0x7f61f8095c20 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.008+0000 7f61f6ffd640 1 -- 172.21.15.177:0/585131621 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0007ac0 con 0x7f61f8095c20 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.008+0000 7f61fe93e640 1 -- 172.21.15.177:0/585131621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095c20 msgr2=0x7f61f81352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.008+0000 7f61fe93e640 1 --2- 172.21.15.177:0/585131621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095c20 0x7f61f81352f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009dc0 tx=0x7f61f002a560 comp rx=0 tx=0).stop 2024-09-13T18:36:24.013 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 -- 172.21.15.177:0/585131621 shutdown_connections 2024-09-13T18:36:24.013 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 --2- 172.21.15.177:0/585131621 >> v2:172.21.15.177:3300/0 conn(0x7f61f8095810 0x7f61f8132da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.013 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 --2- 172.21.15.177:0/585131621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095c20 0x7f61f81352f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.013 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 -- 172.21.15.177:0/585131621 >> 172.21.15.177:0/585131621 conn(0x7f61f809ab20 msgr2=0x7f61f809af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:24.013 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 -- 172.21.15.177:0/585131621 shutdown_connections 2024-09-13T18:36:24.013 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 -- 172.21.15.177:0/585131621 wait complete. 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.009+0000 7f61fe93e640 1 Processor -- start 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 -- start start 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 0x7f61f8124d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095810 0x7f61f8122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61f8095c20 0x7f61f8123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8126400 con 0x7f61f8094b20 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8071630 con 0x7f61f8095c20 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fe93e640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81259e0 con 0x7f61f8095810 2024-09-13T18:36:24.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61f7fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 0x7f61f8124d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61f7fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 0x7f61f8124d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47484/0 (socket says 172.21.15.177:47484) 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61f7fff640 1 -- 172.21.15.177:0/3175215934 learned_addr learned my addr 172.21.15.177:0/3175215934 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61fcedb640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61f8095c20 0x7f61f8123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.010+0000 7f61f77fe640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095810 0x7f61f8122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f7fff640 1 -- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095810 msgr2=0x7f61f8122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f7fff640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095810 0x7f61f8122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f7fff640 1 -- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61f8095c20 msgr2=0x7f61f8123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f7fff640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61f8095c20 0x7f61f8123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f7fff640 1 -- 172.21.15.177:0/3175215934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0009a70 con 0x7f61f8094b20 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61fcedb640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61f8095c20 0x7f61f8123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f7fff640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 0x7f61f8124d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61e800c4a0 tx=0x7f61e800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:24.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80075a0 con 0x7f61f8094b20 2024-09-13T18:36:24.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80047e0 con 0x7f61f8094b20 2024-09-13T18:36:24.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8004c10 con 0x7f61f8094b20 2024-09-13T18:36:24.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.011+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8135240 con 0x7f61f8094b20 2024-09-13T18:36:24.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.012+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f8123920 con 0x7f61f8094b20 2024-09-13T18:36:24.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.013+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8095100 con 0x7f61f8094b20 2024-09-13T18:36:24.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.013+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f61e8012400 con 0x7f61f8094b20 2024-09-13T18:36:24.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.014+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f61e805b970 con 0x7f61f8094b20 2024-09-13T18:36:24.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.015+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc40073cc0 con 0x7fdc40095810 2024-09-13T18:36:24.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.016+0000 7fdc3cff9640 1 -- 172.21.15.177:0/766281274 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdc3003dd70 con 0x7fdc40095810 2024-09-13T18:36:24.020 INFO:teuthology.orchestra.run.smithi177.stdout:38654705681 2024-09-13T18:36:24.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 >> v1:172.21.15.177:6816/516212916 conn(0x7fdc200473b0 legacy=0x7fdc20049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 msgr2=0x7fdc4008c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 0x7fdc4008c460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000ebe0 tx=0x7fdc3000c520 comp rx=0 tx=0).stop 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 shutdown_connections 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc4012b690 0x7fdc401c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc40099db0 0x7fdc4008c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 --2- 172.21.15.177:0/766281274 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdc40095810 0x7fdc4008c460 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 >> 172.21.15.177:0/766281274 conn(0x7fdc4009ab20 msgr2=0x7fdc4009c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 shutdown_connections 2024-09-13T18:36:24.022 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.018+0000 7fdc45e2b640 1 -- 172.21.15.177:0/766281274 wait complete. 2024-09-13T18:36:24.023 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.021+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f61e8039700 con 0x7f61f8094b20 2024-09-13T18:36:24.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.3 2024-09-13T18:36:24.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.110+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8684127460 con 0x7f8684121d70 2024-09-13T18:36:24.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.111+0000 7f866bfff640 1 -- 172.21.15.177:0/3383934148 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f867c03e270 con 0x7f8684121d70 2024-09-13T18:36:24.114 INFO:teuthology.orchestra.run.smithi177.stdout:38654705680 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 >> v1:172.21.15.177:6816/516212916 conn(0x7f86700473b0 legacy=0x7f8670049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 msgr2=0x7f8684122220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 0x7f8684122220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f868406a560 tx=0x7f867c0115a0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 shutdown_connections 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8684121d70 0x7f8684122220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86841206b0 0x7f8684121830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 --2- 172.21.15.177:0/3383934148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868411fd90 0x7f8684120170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.112+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 >> 172.21.15.177:0/3383934148 conn(0x7f868409b680 msgr2=0x7f868412c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.113+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 shutdown_connections 2024-09-13T18:36:24.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.113+0000 7f8669ffb640 1 -- 172.21.15.177:0/3383934148 wait complete. 2024-09-13T18:36:24.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.116+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc90012c1d0 con 0x7fc90012e540 2024-09-13T18:36:24.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.117+0000 7fc8eeffd640 1 -- 172.21.15.177:0/3025830290 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8f8018030 con 0x7fc90012e540 2024-09-13T18:36:24.120 INFO:teuthology.orchestra.run.smithi177.stdout:38654705680 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.118+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 >> v1:172.21.15.177:6816/516212916 conn(0x7fc8e0047430 legacy=0x7fc8e00498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.118+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90012e540 msgr2=0x7fc90012e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.118+0000 7fc9049f4640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90012e540 0x7fc90012e9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009e50 tx=0x7fc8f800c490 comp rx=0 tx=0).stop 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.118+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 shutdown_connections 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.118+0000 7fc9049f4640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90012e540 0x7fc90012e9b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.119+0000 7fc9049f4640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fc90012da20 0x7fc900099bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.119+0000 7fc9049f4640 1 --2- 172.21.15.177:0/3025830290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90006bd60 0x7fc900099680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.119+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 >> 172.21.15.177:0/3025830290 conn(0x7fc90009fe10 msgr2=0x7fc900127530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:24.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.119+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 shutdown_connections 2024-09-13T18:36:24.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.119+0000 7fc9049f4640 1 -- 172.21.15.177:0/3025830290 wait complete. 2024-09-13T18:36:24.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705680 for osd.2 2024-09-13T18:36:24.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705680 for osd.1 2024-09-13T18:36:24.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.163+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f61f8125b50 con 0x7f61f8094b20 2024-09-13T18:36:24.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.164+0000 7f61f4ff9640 1 -- 172.21.15.177:0/3175215934 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f61e8018030 con 0x7f61f8094b20 2024-09-13T18:36:24.167 INFO:teuthology.orchestra.run.smithi177.stdout:38654705681 2024-09-13T18:36:24.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 >> v1:172.21.15.177:6816/516212916 conn(0x7f61e40473b0 legacy=0x7f61e4049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 msgr2=0x7f61f8124d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 0x7f61f8124d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61e800c4a0 tx=0x7f61e800c960 comp rx=0 tx=0).stop 2024-09-13T18:36:24.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 shutdown_connections 2024-09-13T18:36:24.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61f8095c20 0x7f61f8123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f61f8095810 0x7f61f8122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 --2- 172.21.15.177:0/3175215934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f8094b20 0x7f61f8124d60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.165+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 >> 172.21.15.177:0/3175215934 conn(0x7f61f809ab20 msgr2=0x7f61f8091b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:24.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.166+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 shutdown_connections 2024-09-13T18:36:24.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:24.166+0000 7f61fe93e640 1 -- 172.21.15.177:0/3175215934 wait complete. 2024-09-13T18:36:24.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.0 2024-09-13T18:36:25.033 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:25.128 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:25.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 Processor -- start 2024-09-13T18:36:25.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 -- start start 2024-09-13T18:36:25.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda0012a7d0 0x7fda0012abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda00129300 0x7fda00129780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 0x7fda001327a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda000708c0 con 0x7fda0012a7d0 2024-09-13T18:36:25.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0012a380 con 0x7fda00129cc0 2024-09-13T18:36:25.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.129+0000 7fda07a68640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda0006fd10 con 0x7fda00129300 2024-09-13T18:36:25.140 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 0x7fda001327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.141 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 0x7fda001327a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35370/0 (socket says 172.21.15.177:35370) 2024-09-13T18:36:25.141 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 -- 172.21.15.177:0/3321093887 learned_addr learned my addr 172.21.15.177:0/3321093887 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.141 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda05804640 1 --2- 172.21.15.177:0/3321093887 >> v2:172.21.15.177:3300/0 conn(0x7fda0012a7d0 0x7fda0012abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.141 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda05003640 1 --2- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda00129300 0x7fda00129780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.141 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 -- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda00129300 msgr2=0x7fda00129780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 --2- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda00129300 0x7fda00129780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 -- 172.21.15.177:0/3321093887 >> v2:172.21.15.177:3300/0 conn(0x7fda0012a7d0 msgr2=0x7fda0012abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 --2- 172.21.15.177:0/3321093887 >> v2:172.21.15.177:3300/0 conn(0x7fda0012a7d0 0x7fda0012abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda06005640 1 -- 172.21.15.177:0/3321093887 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0012a150 con 0x7fda00129cc0 2024-09-13T18:36:25.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.130+0000 7fda05804640 1 --2- 172.21.15.177:0/3321093887 >> v2:172.21.15.177:3300/0 conn(0x7fda0012a7d0 0x7fda0012abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.131+0000 7fda06005640 1 --2- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 0x7fda001327a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009e50 tx=0x7fd9f802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7324b2180fbf814a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.131+0000 7fda04802640 1 -- 172.21.15.177:0/3321093887 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031600 con 0x7fda00129cc0 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.131+0000 7fda04802640 1 -- 172.21.15.177:0/3321093887 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f802d050 con 0x7fda00129cc0 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.131+0000 7fda04802640 1 -- 172.21.15.177:0/3321093887 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031980 con 0x7fda00129cc0 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 -- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 msgr2=0x7fda001327a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 --2- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 0x7fda001327a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009e50 tx=0x7fd9f802a560 comp rx=0 tx=0).stop 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 -- 172.21.15.177:0/3321093887 shutdown_connections 2024-09-13T18:36:25.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 --2- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129cc0 0x7fda001327a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 --2- 172.21.15.177:0/3321093887 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda00129300 0x7fda00129780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 --2- 172.21.15.177:0/3321093887 >> v2:172.21.15.177:3300/0 conn(0x7fda0012a7d0 0x7fda0012abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 -- 172.21.15.177:0/3321093887 >> 172.21.15.177:0/3321093887 conn(0x7fda0009c990 msgr2=0x7fda0006d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 -- 172.21.15.177:0/3321093887 shutdown_connections 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.132+0000 7fda07a68640 1 -- 172.21.15.177:0/3321093887 wait complete. 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 Processor -- start 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 -- start start 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 0x7fda0011bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda00129cc0 0x7fda0011c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda0012a7d0 0x7fda0011c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0011eb80 con 0x7fda00129cc0 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0011efc0 con 0x7fda00129300 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda07a68640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda0011dfa0 con 0x7fda0012a7d0 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 0x7fda0011bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05003640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda00129cc0 0x7fda0011c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 0x7fda0011bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35386/0 (socket says 172.21.15.177:35386) 2024-09-13T18:36:25.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 -- 172.21.15.177:0/2657195380 learned_addr learned my addr 172.21.15.177:0/2657195380 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda06005640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda0012a7d0 0x7fda0011c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 -- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda0012a7d0 msgr2=0x7fda0011c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda0012a7d0 0x7fda0011c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 -- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda00129cc0 msgr2=0x7fda0011c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda00129cc0 0x7fda0011c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05804640 1 -- 172.21.15.177:0/2657195380 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc009400 con 0x7fda00129300 2024-09-13T18:36:25.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda05003640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda00129cc0 0x7fda0011c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.133+0000 7fda06005640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda0012a7d0 0x7fda0011c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.134+0000 7fda05804640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 0x7fda0011bc10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00ec10 tx=0x7fd9fc00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.134+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc009760 con 0x7fda00129300 2024-09-13T18:36:25.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.134+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc00cc30 con 0x7fda00129300 2024-09-13T18:36:25.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.134+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f8009b30 con 0x7fda00129300 2024-09-13T18:36:25.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.134+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc013470 con 0x7fda00129300 2024-09-13T18:36:25.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.134+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001c03b0 con 0x7fda00129300 2024-09-13T18:36:25.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.135+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fd9fc0135d0 con 0x7fda00129300 2024-09-13T18:36:25.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.135+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00129780 con 0x7fda00129300 2024-09-13T18:36:25.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.135+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fd9fc05caf0 con 0x7fda00129300 2024-09-13T18:36:25.149 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:25.150 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.139+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fd9fc03a780 con 0x7fda00129300 2024-09-13T18:36:25.180 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 Processor -- start 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 -- start start 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9af412da20 0x7f9af412dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af406d840 con 0x7f9af412da20 2024-09-13T18:36:25.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af406c4a0 con 0x7f9af406bd60 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35396/0 (socket says 172.21.15.177:35396) 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af259c640 1 -- 172.21.15.177:0/93095244 learned_addr learned my addr 172.21.15.177:0/93095244 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af9176640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9af406db60 con 0x7f9af406b440 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.228+0000 7f9af359e640 1 --2- 172.21.15.177:0/93095244 >> v2:172.21.15.177:3300/0 conn(0x7f9af412da20 0x7f9af412dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.229+0000 7f9af2d9d640 1 -- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 msgr2=0x7f9af412d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.229+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.229+0000 7f9af2d9d640 1 -- 172.21.15.177:0/93095244 >> v2:172.21.15.177:3300/0 conn(0x7f9af412da20 msgr2=0x7f9af412dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.229+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/93095244 >> v2:172.21.15.177:3300/0 conn(0x7f9af412da20 0x7f9af412dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.229+0000 7f9af2d9d640 1 -- 172.21.15.177:0/93095244 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af406c140 con 0x7f9af406b440 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.230+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af406b820 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40097a0 tx=0x7f9ae4029fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f94804b1e941a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.230+0000 7f9af1d9b640 1 -- 172.21.15.177:0/93095244 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae402ad30 con 0x7f9af406b440 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.230+0000 7f9af1d9b640 1 -- 172.21.15.177:0/93095244 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae402ae90 con 0x7f9af406b440 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.230+0000 7f9af1d9b640 1 -- 172.21.15.177:0/93095244 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4032660 con 0x7f9af406b440 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 -- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 msgr2=0x7f9af406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 --2- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af406b820 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40097a0 tx=0x7f9ae4029fe0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 -- 172.21.15.177:0/93095244 shutdown_connections 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 --2- 172.21.15.177:0/93095244 >> v2:172.21.15.177:3300/0 conn(0x7f9af412da20 0x7f9af412dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 --2- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 --2- 172.21.15.177:0/93095244 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af406b820 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 -- 172.21.15.177:0/93095244 >> 172.21.15.177:0/93095244 conn(0x7f9af409fe10 msgr2=0x7f9af40a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 -- 172.21.15.177:0/93095244 shutdown_connections 2024-09-13T18:36:25.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.231+0000 7f9af9176640 1 -- 172.21.15.177:0/93095244 wait complete. 2024-09-13T18:36:25.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.233+0000 7f9af9176640 1 Processor -- start 2024-09-13T18:36:25.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.233+0000 7f9af9176640 1 -- start start 2024-09-13T18:36:25.237 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.233+0000 7f9af9176640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af412e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.237 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.233+0000 7f9af9176640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.237 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af9176640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 0x7f9af412f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af412e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af412e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:33020/0 (socket says 172.21.15.177:33020) 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af359e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 0x7f9af412f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af359e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 0x7f9af412f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47544/0 (socket says 172.21.15.177:47544) 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 -- 172.21.15.177:0/556135408 learned_addr learned my addr 172.21.15.177:0/556135408 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4130c10 con 0x7f9af412da20 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4131830 con 0x7f9af406bd60 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9af412fde0 con 0x7f9af406b440 2024-09-13T18:36:25.238 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af259c640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 -- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 msgr2=0x7f9af412ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 -- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 msgr2=0x7f9af412f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 0x7f9af412f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4009400 con 0x7f9af406b440 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af359e640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 0x7f9af412f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.234+0000 7f9af2d9d640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af412e840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40098c0 tx=0x7f9ae40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.235+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae403b600 con 0x7f9af406b440 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.235+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae403b760 con 0x7f9af406b440 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.235+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af4131520 con 0x7f9af406b440 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.235+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af4134540 con 0x7f9af406b440 2024-09-13T18:36:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.235+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae403bb10 con 0x7f9af406b440 2024-09-13T18:36:25.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.236+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f9ae4039040 con 0x7f9af406b440 2024-09-13T18:36:25.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.236+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f9ae407ca00 con 0x7f9af406b440 2024-09-13T18:36:25.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.236+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af412d4e0 con 0x7f9af406b440 2024-09-13T18:36:25.247 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.244+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f9ae4037030 con 0x7f9af406b440 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 Processor -- start 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 -- start start 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e1806bf30 0x7f5e18132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e18132d40 0x7f5e18135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1806d840 con 0x7f5e1806b440 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1806c4a0 con 0x7f5e1806bf30 2024-09-13T18:36:25.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.256+0000 7f5e1e23d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1806db60 con 0x7f5e18132d40 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e177fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5e1806b440 0x7f5e1806b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58184/0 (socket says 172.21.15.177:58184) 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e17fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e18132d40 0x7f5e18135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e177fe640 1 -- 172.21.15.177:0/2382807889 learned_addr learned my addr 172.21.15.177:0/2382807889 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e16ffd640 1 --2- 172.21.15.177:0/2382807889 >> v2:172.21.15.177:3300/0 conn(0x7f5e1806bf30 0x7f5e18132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e167fc640 1 -- 172.21.15.177:0/2382807889 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670068937 0 0) 0x7f5e1806d840 con 0x7f5e1806b440 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e167fc640 1 -- 172.21.15.177:0/2382807889 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e00003520 con 0x7f5e1806b440 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e17fff640 1 -- 172.21.15.177:0/2382807889 >> v2:172.21.15.177:3300/0 conn(0x7f5e1806bf30 msgr2=0x7f5e18132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e17fff640 1 --2- 172.21.15.177:0/2382807889 >> v2:172.21.15.177:3300/0 conn(0x7f5e1806bf30 0x7f5e18132800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e17fff640 1 -- 172.21.15.177:0/2382807889 >> v1:172.21.15.177:6789/0 conn(0x7f5e1806b440 legacy=0x7f5e1806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.261 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.257+0000 7f5e17fff640 1 -- 172.21.15.177:0/2382807889 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1806cfb0 con 0x7f5e18132d40 2024-09-13T18:36:25.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.258+0000 7f5e17fff640 1 --2- 172.21.15.177:0/2382807889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e18132d40 0x7f5e18135130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=749e2f9e28195d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.258+0000 7f5e167fc640 1 -- 172.21.15.177:0/2382807889 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10007740 con 0x7f5e18132d40 2024-09-13T18:36:25.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.258+0000 7f5e167fc640 1 -- 172.21.15.177:0/2382807889 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1002d050 con 0x7f5e18132d40 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.258+0000 7f5e167fc640 1 -- 172.21.15.177:0/2382807889 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10007ac0 con 0x7f5e18132d40 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 -- 172.21.15.177:0/2382807889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e18132d40 msgr2=0x7f5e18135130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/2382807889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e18132d40 0x7f5e18135130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1002a530 comp rx=0 tx=0).stop 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 -- 172.21.15.177:0/2382807889 shutdown_connections 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/2382807889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e18132d40 0x7f5e18135130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/2382807889 >> v2:172.21.15.177:3300/0 conn(0x7f5e1806bf30 0x7f5e18132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 -- 172.21.15.177:0/2382807889 >> 172.21.15.177:0/2382807889 conn(0x7f5e1809fe10 msgr2=0x7f5e180a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.259+0000 7f5e1e23d640 1 -- 172.21.15.177:0/2382807889 shutdown_connections 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.260+0000 7f5e1e23d640 1 -- 172.21.15.177:0/2382807889 wait complete. 2024-09-13T18:36:25.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.261+0000 7f5e1e23d640 1 Processor -- start 2024-09-13T18:36:25.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.261+0000 7f5e1e23d640 1 -- start start 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e1e23d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e1806b440 0x7f5e1806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e1e23d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 0x7f5e1806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e1e23d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e18132d40 0x7f5e1806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e1e23d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18070950 con 0x7f5e1806b440 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e1e23d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18070170 con 0x7f5e1806bf30 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e1e23d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1806f930 con 0x7f5e18132d40 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 0x7f5e1806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 0x7f5e1806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35422/0 (socket says 172.21.15.177:35422) 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 -- 172.21.15.177:0/302062376 learned_addr learned my addr 172.21.15.177:0/302062376 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e17fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e18132d40 0x7f5e1806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e177fe640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e1806b440 0x7f5e1806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 -- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e18132d40 msgr2=0x7f5e1806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e18132d40 0x7f5e1806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 -- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e1806b440 msgr2=0x7f5e1806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e1806b440 0x7f5e1806de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 -- 172.21.15.177:0/302062376 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009a70 con 0x7f5e1806bf30 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e177fe640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e1806b440 0x7f5e1806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.262+0000 7f5e16ffd640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 0x7f5e1806e360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00e9e0 tx=0x7f5e0c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.263+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c009670 con 0x7f5e1806bf30 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.263+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1806f020 con 0x7f5e1806bf30 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.263+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0c00c9d0 con 0x7f5e1806bf30 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.263+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00cd80 con 0x7f5e1806bf30 2024-09-13T18:36:25.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.263+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e18132500 con 0x7f5e1806bf30 2024-09-13T18:36:25.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.264+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1806b820 con 0x7f5e1806bf30 2024-09-13T18:36:25.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.264+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f5e0c013470 con 0x7f5e1806bf30 2024-09-13T18:36:25.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.265+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f5e0c05cae0 con 0x7f5e1806bf30 2024-09-13T18:36:25.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.272+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f5e0c03a6d0 con 0x7f5e1806bf30 2024-09-13T18:36:25.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.288+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fda0006a620 con 0x7fda00129300 2024-09-13T18:36:25.292 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.289+0000 7fd9f67fc640 1 -- 172.21.15.177:0/2657195380 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd9fc03dda0 con 0x7fda00129300 2024-09-13T18:36:25.292 INFO:teuthology.orchestra.run.smithi177.stdout:38654705684 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.290+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 >> v1:172.21.15.177:6816/516212916 conn(0x7fd9e40474d0 legacy=0x7fd9e4049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.290+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 msgr2=0x7fda0011bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.290+0000 7fda07a68640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 0x7fda0011bc10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00ec10 tx=0x7fd9fc00c520 comp rx=0 tx=0).stop 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 shutdown_connections 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fda0012a7d0 0x7fda0011c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda00129cc0 0x7fda0011c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 --2- 172.21.15.177:0/2657195380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda00129300 0x7fda0011bc10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 >> 172.21.15.177:0/2657195380 conn(0x7fda0009c990 msgr2=0x7fda00114bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 shutdown_connections 2024-09-13T18:36:25.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.291+0000 7fda07a68640 1 -- 172.21.15.177:0/2657195380 wait complete. 2024-09-13T18:36:25.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.293+0000 7f9156cc9640 1 Processor -- start 2024-09-13T18:36:25.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.293+0000 7f9156cc9640 1 -- start start 2024-09-13T18:36:25.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9156cc9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9150099db0 0x7f91501305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9156cc9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f9150132ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9156cc9640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915012c180 con 0x7f915012b690 2024-09-13T18:36:25.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9156cc9640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9150096a20 con 0x7f9150099db0 2024-09-13T18:36:25.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9156cc9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9150096cf0 con 0x7f9150130b00 2024-09-13T18:36:25.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9155266640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f9150132ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9154a65640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f915012b690 0x7f915012ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58194/0 (socket says 172.21.15.177:58194) 2024-09-13T18:36:25.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f9154a65640 1 -- 172.21.15.177:0/920102472 learned_addr learned my addr 172.21.15.177:0/920102472 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:25.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f914ffff640 1 --2- 172.21.15.177:0/920102472 >> v2:172.21.15.177:3300/0 conn(0x7f9150099db0 0x7f91501305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f914ffff640 1 -- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 msgr2=0x7f9150132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f914ffff640 1 --2- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f9150132ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f914f7fe640 1 -- 172.21.15.177:0/920102472 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133701270 0 0) 0x7f915012c180 con 0x7f915012b690 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f914ffff640 1 -- 172.21.15.177:0/920102472 >> v1:172.21.15.177:6789/0 conn(0x7f915012b690 legacy=0x7f915012ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.294+0000 7f914ffff640 1 -- 172.21.15.177:0/920102472 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500961b0 con 0x7f9150099db0 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.296+0000 7f914ffff640 1 --2- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9150099db0 0x7f91501305c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f913c0097a0 tx=0x7f913c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99acddc5617694c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.296+0000 7f914f7fe640 1 -- 172.21.15.177:0/920102472 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c02adb0 con 0x7f9150099db0 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.296+0000 7f914f7fe640 1 -- 172.21.15.177:0/920102472 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c032440 con 0x7f9150099db0 2024-09-13T18:36:25.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.296+0000 7f914f7fe640 1 -- 172.21.15.177:0/920102472 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0327d0 con 0x7f9150099db0 2024-09-13T18:36:25.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 -- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9150099db0 msgr2=0x7f91501305c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 --2- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9150099db0 0x7f91501305c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f913c0097a0 tx=0x7f913c02a060 comp rx=0 tx=0).stop 2024-09-13T18:36:25.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 -- 172.21.15.177:0/920102472 shutdown_connections 2024-09-13T18:36:25.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 --2- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9150099db0 0x7f91501305c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 --2- 172.21.15.177:0/920102472 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f9150132ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 -- 172.21.15.177:0/920102472 >> 172.21.15.177:0/920102472 conn(0x7f915009ab20 msgr2=0x7f915009af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 -- 172.21.15.177:0/920102472 shutdown_connections 2024-09-13T18:36:25.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.297+0000 7f9156cc9640 1 -- 172.21.15.177:0/920102472 wait complete. 2024-09-13T18:36:25.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.298+0000 7f9156cc9640 1 Processor -- start 2024-09-13T18:36:25.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.298+0000 7f9156cc9640 1 -- start start 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9156cc9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9150099db0 0x7f9150070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9156cc9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 0x7f9150070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9156cc9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f915006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9156cc9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915006f5d0 con 0x7f915012b690 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9156cc9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9150071790 con 0x7f9150099db0 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9156cc9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9150070d80 con 0x7f9150130b00 2024-09-13T18:36:25.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f914ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 0x7f9150070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f914ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 0x7f9150070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47592/0 (socket says 172.21.15.177:47592) 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f914ffff640 1 -- 172.21.15.177:0/2736698635 learned_addr learned my addr 172.21.15.177:0/2736698635 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9155266640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f915006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9154a65640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9150099db0 0x7f9150070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9155266640 1 -- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9150099db0 msgr2=0x7f9150070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9155266640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9150099db0 0x7f9150070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9155266640 1 -- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 msgr2=0x7f9150070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9155266640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 0x7f9150070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9155266640 1 -- 172.21.15.177:0/2736698635 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009400 con 0x7f9150130b00 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f9154a65640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9150099db0 0x7f9150070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.299+0000 7f914ffff640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 0x7f9150070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.300+0000 7f9155266640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f915006c300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f914800ef90 tx=0x7f914800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.300+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148009c00 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.300+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800f040 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.300+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915006dcc0 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.300+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480146c0 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.300+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915006fc60 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.301+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f9148014820 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.301+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915006ff40 con 0x7f9150130b00 2024-09-13T18:36:25.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.302+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f914805d580 con 0x7f9150130b00 2024-09-13T18:36:25.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705684 for osd.3 2024-09-13T18:36:25.307 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:25.309 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.306+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f914803b270 con 0x7f9150130b00 2024-09-13T18:36:25.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.400+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9af41265e0 con 0x7f9af406b440 2024-09-13T18:36:25.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.401+0000 7f9adb7fe640 1 -- 172.21.15.177:0/556135408 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ae4037030 con 0x7f9af406b440 2024-09-13T18:36:25.404 INFO:teuthology.orchestra.run.smithi177.stdout:38654705683 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 >> v1:172.21.15.177:6816/516212916 conn(0x7f9ae00474d0 legacy=0x7f9ae0049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 msgr2=0x7f9af412e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af412e840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40098c0 tx=0x7f9ae40046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 shutdown_connections 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af412da20 0x7f9af412f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af406bd60 0x7f9af412ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.402+0000 7f9af9176640 1 --2- 172.21.15.177:0/556135408 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9af406b440 0x7f9af412e840 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.403+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 >> 172.21.15.177:0/556135408 conn(0x7f9af409fe10 msgr2=0x7f9af409bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.403+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 shutdown_connections 2024-09-13T18:36:25.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.403+0000 7f9af9176640 1 -- 172.21.15.177:0/556135408 wait complete. 2024-09-13T18:36:25.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705683 for osd.2 2024-09-13T18:36:25.417 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:25.434 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.431+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5e181265e0 con 0x7f5e1806bf30 2024-09-13T18:36:25.434 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.431+0000 7f5dfffff640 1 -- 172.21.15.177:0/302062376 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5e0c03dcf0 con 0x7f5e1806bf30 2024-09-13T18:36:25.435 INFO:teuthology.orchestra.run.smithi177.stdout:38654705683 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 >> v1:172.21.15.177:6816/516212916 conn(0x7f5e040474d0 legacy=0x7f5e04049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 msgr2=0x7f5e1806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 0x7f5e1806e360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00e9e0 tx=0x7f5e0c00eea0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 shutdown_connections 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5e18132d40 0x7f5e1806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e1806bf30 0x7f5e1806e360 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 --2- 172.21.15.177:0/302062376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e1806b440 0x7f5e1806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 >> 172.21.15.177:0/302062376 conn(0x7f5e1809fe10 msgr2=0x7f5e1809bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 shutdown_connections 2024-09-13T18:36:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.433+0000 7f5e1e23d640 1 -- 172.21.15.177:0/302062376 wait complete. 2024-09-13T18:36:25.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705683 for osd.1 2024-09-13T18:36:25.448 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:25.463 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.460+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f915006f9a0 con 0x7f9150130b00 2024-09-13T18:36:25.463 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.460+0000 7f914d7fa640 1 -- 172.21.15.177:0/2736698635 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9148018070 con 0x7f9150130b00 2024-09-13T18:36:25.463 INFO:teuthology.orchestra.run.smithi177.stdout:38654705684 2024-09-13T18:36:25.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 >> v1:172.21.15.177:6816/516212916 conn(0x7f9134047500 legacy=0x7f91340499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 msgr2=0x7f915006c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f915006c300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f914800ef90 tx=0x7f914800c490 comp rx=0 tx=0).stop 2024-09-13T18:36:25.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 shutdown_connections 2024-09-13T18:36:25.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9150130b00 0x7f915006c300 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f915012b690 0x7f9150070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.466 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 --2- 172.21.15.177:0/2736698635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9150099db0 0x7f9150070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.466 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 >> 172.21.15.177:0/2736698635 conn(0x7f915009ab20 msgr2=0x7f9150095100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.466 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 shutdown_connections 2024-09-13T18:36:25.466 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.462+0000 7f9156cc9640 1 -- 172.21.15.177:0/2736698635 wait complete. 2024-09-13T18:36:25.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705684 for osd.0 2024-09-13T18:36:25.476 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:25.476 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:36:25.476 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 Processor -- start 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 -- start start 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7860095270 0x7f7860095650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095b90 0x7f786006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786006e080 con 0x7f786006cca0 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786006db30 con 0x7f7860095270 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7867244640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7860096000 con 0x7f7860095b90 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f7864fe0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7860095270 0x7f7860095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f78657e1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095b90 0x7f786006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f785ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f786006cca0 0x7f786006b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58206/0 (socket says 172.21.15.177:58206) 2024-09-13T18:36:25.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.612+0000 7f785ffff640 1 -- 172.21.15.177:0/4104868859 learned_addr learned my addr 172.21.15.177:0/4104868859 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f785f7fe640 1 -- 172.21.15.177:0/4104868859 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803440842 0 0) 0x7f786006e080 con 0x7f786006cca0 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f78657e1640 1 -- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7860095270 msgr2=0x7f7860095650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f78657e1640 1 --2- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7860095270 0x7f7860095650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f78657e1640 1 -- 172.21.15.177:0/4104868859 >> v1:172.21.15.177:6789/0 conn(0x7f786006cca0 legacy=0x7f786006b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f78657e1640 1 -- 172.21.15.177:0/4104868859 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786006e390 con 0x7f7860095b90 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f7864fe0640 1 --2- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7860095270 0x7f7860095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.613+0000 7f78657e1640 1 --2- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095b90 0x7f786006b8e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f785800be60 tx=0x7f785800a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b4d468bcf711aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f785f7fe640 1 -- 172.21.15.177:0/4104868859 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858030480 con 0x7f7860095b90 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f785f7fe640 1 -- 172.21.15.177:0/4104868859 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785802d030 con 0x7f7860095b90 2024-09-13T18:36:25.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f785f7fe640 1 -- 172.21.15.177:0/4104868859 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858030800 con 0x7f7860095b90 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 -- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095b90 msgr2=0x7f786006b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 --2- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095b90 0x7f786006b8e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f785800be60 tx=0x7f785800a400 comp rx=0 tx=0).stop 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 -- 172.21.15.177:0/4104868859 shutdown_connections 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 --2- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095b90 0x7f786006b8e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 --2- 172.21.15.177:0/4104868859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7860095270 0x7f7860095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 -- 172.21.15.177:0/4104868859 >> 172.21.15.177:0/4104868859 conn(0x7f7860098ba0 msgr2=0x7f7860096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 -- 172.21.15.177:0/4104868859 shutdown_connections 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.614+0000 7f7867244640 1 -- 172.21.15.177:0/4104868859 wait complete. 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.615+0000 7f7867244640 1 Processor -- start 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.615+0000 7f7867244640 1 -- start start 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 0x7f78601be050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095270 0x7f78601be590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7860095b90 0x7f7860092da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78601216e0 con 0x7f7860095b90 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7860122100 con 0x7f786006cca0 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7860121d30 con 0x7f7860095270 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7864fe0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 0x7f78601be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095270 0x7f78601be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7864fe0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 0x7f78601be050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35446/0 (socket says 172.21.15.177:35446) 2024-09-13T18:36:25.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7864fe0640 1 -- 172.21.15.177:0/942688602 learned_addr learned my addr 172.21.15.177:0/942688602 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f78657e1640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7860095b90 0x7f7860092da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 -- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 msgr2=0x7f78601be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 0x7f78601be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 -- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7860095b90 msgr2=0x7f7860092da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7860095b90 0x7f7860092da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 -- 172.21.15.177:0/942688602 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785400d8a0 con 0x7f7860095270 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7864fe0640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 0x7f78601be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f78657e1640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7860095b90 0x7f7860092da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785ffff640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095270 0x7f78601be590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c3e0 tx=0x7f784c00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0074e0 con 0x7f7860095270 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c007640 con 0x7f7860095270 2024-09-13T18:36:25.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785800ba90 con 0x7f7860095270 2024-09-13T18:36:25.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.616+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c004a20 con 0x7f7860095270 2024-09-13T18:36:25.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.617+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7860094e40 con 0x7f7860095270 2024-09-13T18:36:25.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.617+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860095650 con 0x7f7860095270 2024-09-13T18:36:25.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.617+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f784c004b80 con 0x7f7860095270 2024-09-13T18:36:25.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.618+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f784c05c510 con 0x7f7860095270 2024-09-13T18:36:25.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.621+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f784c03a200 con 0x7f7860095270 2024-09-13T18:36:25.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.752+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78601beb90 con 0x7f7844047500 2024-09-13T18:36:25.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.753+0000 7f785d7fa640 1 -- 172.21.15.177:0/942688602 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40159 (unknown 2965378022 0 1111276449) 0x7f78601beb90 con 0x7f7844047500 2024-09-13T18:36:25.756 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:25.757 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.754+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 >> v1:172.21.15.177:6816/516212916 conn(0x7f7844047500 legacy=0x7f78440499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.754+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095270 msgr2=0x7f78601be590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.754+0000 7f7867244640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095270 0x7f78601be590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c3e0 tx=0x7f784c00c8a0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 shutdown_connections 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7860095b90 0x7f7860092da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f7860095270 0x7f78601be590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 --2- 172.21.15.177:0/942688602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f786006cca0 0x7f78601be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 >> 172.21.15.177:0/942688602 conn(0x7f7860098ba0 msgr2=0x7f7860099950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 shutdown_connections 2024-09-13T18:36:25.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.755+0000 7f7867244640 1 -- 172.21.15.177:0/942688602 wait complete. 2024-09-13T18:36:25.768 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-13T18:36:24.590762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191260,"kb_used_data":1836,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296100,"statfs":{"total":386547056640,"available":385327206400,"internally_reserved":0,"allocated":1880064,"data_stored":1382343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.000820"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985562+0000","last_change":"2024-09-13T18:35:13.735440+0000","last_active":"2024-09-13T18:35:18.985562+0000","last_peered":"2024-09-13T18:35:18.985562+0000","last_clean":"2024-09-13T18:35:18.985562+0000","last_became_active":"2024-09-13T18:35:11.730650+0000","last_became_peered":"2024-09-13T18:35:11.730650+0000","last_unstale":"2024-09-13T18:35:18.985562+0000","last_undegraded":"2024-09-13T18:35:18.985562+0000","last_fullsized":"2024-09-13T18:35:18.985562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:35.164028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.986368+0000","last_change":"2024-09-13T18:35:13.735179+0000","last_active":"2024-09-13T18:35:18.986368+0000","last_peered":"2024-09-13T18:35:18.986368+0000","last_clean":"2024-09-13T18:35:18.986368+0000","last_became_active":"2024-09-13T18:35:11.730100+0000","last_became_peered":"2024-09-13T18:35:11.730100+0000","last_unstale":"2024-09-13T18:35:18.986368+0000","last_undegraded":"2024-09-13T18:35:18.986368+0000","last_fullsized":"2024-09-13T18:35:18.986368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:25.061944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985362+0000","last_change":"2024-09-13T18:35:13.735829+0000","last_active":"2024-09-13T18:35:18.985362+0000","last_peered":"2024-09-13T18:35:18.985362+0000","last_clean":"2024-09-13T18:35:18.985362+0000","last_became_active":"2024-09-13T18:35:11.730242+0000","last_became_peered":"2024-09-13T18:35:11.730242+0000","last_unstale":"2024-09-13T18:35:18.985362+0000","last_undegraded":"2024-09-13T18:35:18.985362+0000","last_fullsized":"2024-09-13T18:35:18.985362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:01:03.137288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.986335+0000","last_change":"2024-09-13T18:35:13.735117+0000","last_active":"2024-09-13T18:35:18.986335+0000","last_peered":"2024-09-13T18:35:18.986335+0000","last_clean":"2024-09-13T18:35:18.986335+0000","last_became_active":"2024-09-13T18:35:11.730015+0000","last_became_peered":"2024-09-13T18:35:11.730015+0000","last_unstale":"2024-09-13T18:35:18.986335+0000","last_undegraded":"2024-09-13T18:35:18.986335+0000","last_fullsized":"2024-09-13T18:35:18.986335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:43.650449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985381+0000","last_change":"2024-09-13T18:35:13.735587+0000","last_active":"2024-09-13T18:35:18.985381+0000","last_peered":"2024-09-13T18:35:18.985381+0000","last_clean":"2024-09-13T18:35:18.985381+0000","last_became_active":"2024-09-13T18:35:11.733034+0000","last_became_peered":"2024-09-13T18:35:11.733034+0000","last_unstale":"2024-09-13T18:35:18.985381+0000","last_undegraded":"2024-09-13T18:35:18.985381+0000","last_fullsized":"2024-09-13T18:35:18.985381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:01:22.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:19.347949+0000","last_change":"2024-09-13T18:35:14.461158+0000","last_active":"2024-09-13T18:35:19.347949+0000","last_peered":"2024-09-13T18:35:19.347949+0000","last_clean":"2024-09-13T18:35:19.347949+0000","last_became_active":"2024-09-13T18:35:11.729626+0000","last_became_peered":"2024-09-13T18:35:11.729626+0000","last_unstale":"2024-09-13T18:35:19.347949+0000","last_undegraded":"2024-09-13T18:35:19.347949+0000","last_fullsized":"2024-09-13T18:35:19.347949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:10.111028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985349+0000","last_change":"2024-09-13T18:35:13.735139+0000","last_active":"2024-09-13T18:35:18.985349+0000","last_peered":"2024-09-13T18:35:18.985349+0000","last_clean":"2024-09-13T18:35:18.985349+0000","last_became_active":"2024-09-13T18:35:11.733025+0000","last_became_peered":"2024-09-13T18:35:11.733025+0000","last_unstale":"2024-09-13T18:35:18.985349+0000","last_undegraded":"2024-09-13T18:35:18.985349+0000","last_fullsized":"2024-09-13T18:35:18.985349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.647913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985651+0000","last_change":"2024-09-13T18:35:13.735186+0000","last_active":"2024-09-13T18:35:18.985651+0000","last_peered":"2024-09-13T18:35:18.985651+0000","last_clean":"2024-09-13T18:35:18.985651+0000","last_became_active":"2024-09-13T18:35:11.727196+0000","last_became_peered":"2024-09-13T18:35:11.727196+0000","last_unstale":"2024-09-13T18:35:18.985651+0000","last_undegraded":"2024-09-13T18:35:18.985651+0000","last_fullsized":"2024-09-13T18:35:18.985651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:18.070546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":82,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.986588+0000","last_change":"2024-09-13T18:35:07.177729+0000","last_active":"2024-09-13T18:35:18.986588+0000","last_peered":"2024-09-13T18:35:18.986588+0000","last_clean":"2024-09-13T18:35:18.986588+0000","last_became_active":"2024-09-13T18:35:07.176786+0000","last_became_peered":"2024-09-13T18:35:07.176786+0000","last_unstale":"2024-09-13T18:35:18.986588+0000","last_undegraded":"2024-09-13T18:35:18.986588+0000","last_fullsized":"2024-09-13T18:35:18.986588+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:02.631221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":50564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.441},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.501},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"last":0.47099999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":172032,"data_stored":50545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.438}]},{"osd":3,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.495},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004}]}]},{"osd":1,"up_from":9,"seq":38654705683,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":770048,"data_stored":640932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.309},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.27100000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29699999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705684,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":761856,"data_stored":640302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:25.770 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:25.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 Processor -- start 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 -- start start 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8ca406b1d0 0x7f8ca406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 0x7f8ca4095000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca406fb40 con 0x7f8ca4095540 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca4097e00 con 0x7f8ca406b1d0 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca8a96640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca406fd70 con 0x7f8ca406baf0 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 0x7f8ca4095000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8ca259c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8ca406b1d0 0x7f8ca406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 0x7f8ca4095000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35452/0 (socket says 172.21.15.177:35452) 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.863+0000 7f8c9bfff640 1 -- 172.21.15.177:0/3149751491 learned_addr learned my addr 172.21.15.177:0/3149751491 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca259c640 1 -- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 msgr2=0x7f8ca4095000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca259c640 1 --2- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 0x7f8ca4095000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca259c640 1 -- 172.21.15.177:0/3149751491 >> v1:172.21.15.177:6789/0 conn(0x7f8ca4095540 legacy=0x7f8ca4132780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca259c640 1 -- 172.21.15.177:0/3149751491 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca406f1c0 con 0x7f8ca406b1d0 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca1d9b640 1 -- 172.21.15.177:0/3149751491 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597969153 0 0) 0x7f8ca406fb40 con 0x7f8ca4095540 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8c9bfff640 1 --2- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 0x7f8ca4095000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca259c640 1 --2- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca406b1d0 0x7f8ca406b5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097a0 tx=0x7f8c9402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dba51e24fd5dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca1d9b640 1 -- 172.21.15.177:0/3149751491 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9402adb0 con 0x7f8ca406b1d0 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.864+0000 7f8ca1d9b640 1 -- 172.21.15.177:0/3149751491 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94032440 con 0x7f8ca406b1d0 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca1d9b640 1 -- 172.21.15.177:0/3149751491 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940327d0 con 0x7f8ca406b1d0 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca406b1d0 msgr2=0x7f8ca406b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca406b1d0 0x7f8ca406b5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097a0 tx=0x7f8c9402a060 comp rx=0 tx=0).stop 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3149751491 shutdown_connections 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406baf0 0x7f8ca4095000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3149751491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca406b1d0 0x7f8ca406b5b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3149751491 >> 172.21.15.177:0/3149751491 conn(0x7f8ca4098ba0 msgr2=0x7f8ca4096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3149751491 shutdown_connections 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.865+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3149751491 wait complete. 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 Processor -- start 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 -- start start 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 0x7f8ca406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8ca406baf0 0x7f8ca406c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca4095540 0x7f8ca406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca406dc80 con 0x7f8ca4095540 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca406e220 con 0x7f8ca406b1d0 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.866+0000 7f8ca8a96640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca406d360 con 0x7f8ca406baf0 2024-09-13T18:36:25.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 0x7f8ca406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca2d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca4095540 0x7f8ca406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8ca406baf0 0x7f8ca406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 0x7f8ca406c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35460/0 (socket says 172.21.15.177:35460) 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 -- 172.21.15.177:0/3295295141 learned_addr learned my addr 172.21.15.177:0/3295295141 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 -- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8ca406baf0 msgr2=0x7f8ca406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8ca406baf0 0x7f8ca406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 -- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca4095540 msgr2=0x7f8ca406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca4095540 0x7f8ca406cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 -- 172.21.15.177:0/3295295141 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c0094d0 con 0x7f8ca406b1d0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca259c640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 0x7f8ca406c2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c9402a580 tx=0x7f8c940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9403d850 con 0x7f8ca406b1d0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9403d9b0 con 0x7f8ca406b1d0 2024-09-13T18:36:25.874 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c94009400 con 0x7f8ca406b1d0 2024-09-13T18:36:25.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9403dd60 con 0x7f8ca406b1d0 2024-09-13T18:36:25.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.867+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca4125880 con 0x7f8ca406b1d0 2024-09-13T18:36:25.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.868+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f8c94039040 con 0x7f8ca406b1d0 2024-09-13T18:36:25.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.868+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca4124af0 con 0x7f8ca406b1d0 2024-09-13T18:36:25.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.868+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f8c9407d160 con 0x7f8ca406b1d0 2024-09-13T18:36:25.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:25.872+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f8c94047d00 con 0x7f8ca406b1d0 2024-09-13T18:36:26.008 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.002+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ca409aff0 con 0x7f8c840474d0 2024-09-13T18:36:26.009 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.004+0000 7f8c9affd640 1 -- 172.21.15.177:0/3295295141 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40159 (unknown 2965378022 0 1111276449) 0x7f8ca409aff0 con 0x7f8c840474d0 2024-09-13T18:36:26.009 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:26.010 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:26.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 >> v1:172.21.15.177:6816/516212916 conn(0x7f8c840474d0 legacy=0x7f8c84049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 msgr2=0x7f8ca406c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 0x7f8ca406c2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c9402a580 tx=0x7f8c940046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 shutdown_connections 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca4095540 0x7f8ca406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f8ca406baf0 0x7f8ca406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 --2- 172.21.15.177:0/3295295141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca406b1d0 0x7f8ca406c2e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 >> 172.21.15.177:0/3295295141 conn(0x7f8ca4098ba0 msgr2=0x7f8ca4125d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 shutdown_connections 2024-09-13T18:36:26.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.005+0000 7f8ca8a96640 1 -- 172.21.15.177:0/3295295141 wait complete. 2024-09-13T18:36:26.019 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-13T18:36:24.590762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191260,"kb_used_data":1836,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296100,"statfs":{"total":386547056640,"available":385327206400,"internally_reserved":0,"allocated":1880064,"data_stored":1382343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.000820"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985562+0000","last_change":"2024-09-13T18:35:13.735440+0000","last_active":"2024-09-13T18:35:18.985562+0000","last_peered":"2024-09-13T18:35:18.985562+0000","last_clean":"2024-09-13T18:35:18.985562+0000","last_became_active":"2024-09-13T18:35:11.730650+0000","last_became_peered":"2024-09-13T18:35:11.730650+0000","last_unstale":"2024-09-13T18:35:18.985562+0000","last_undegraded":"2024-09-13T18:35:18.985562+0000","last_fullsized":"2024-09-13T18:35:18.985562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:35.164028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.986368+0000","last_change":"2024-09-13T18:35:13.735179+0000","last_active":"2024-09-13T18:35:18.986368+0000","last_peered":"2024-09-13T18:35:18.986368+0000","last_clean":"2024-09-13T18:35:18.986368+0000","last_became_active":"2024-09-13T18:35:11.730100+0000","last_became_peered":"2024-09-13T18:35:11.730100+0000","last_unstale":"2024-09-13T18:35:18.986368+0000","last_undegraded":"2024-09-13T18:35:18.986368+0000","last_fullsized":"2024-09-13T18:35:18.986368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:25.061944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985362+0000","last_change":"2024-09-13T18:35:13.735829+0000","last_active":"2024-09-13T18:35:18.985362+0000","last_peered":"2024-09-13T18:35:18.985362+0000","last_clean":"2024-09-13T18:35:18.985362+0000","last_became_active":"2024-09-13T18:35:11.730242+0000","last_became_peered":"2024-09-13T18:35:11.730242+0000","last_unstale":"2024-09-13T18:35:18.985362+0000","last_undegraded":"2024-09-13T18:35:18.985362+0000","last_fullsized":"2024-09-13T18:35:18.985362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:01:03.137288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.986335+0000","last_change":"2024-09-13T18:35:13.735117+0000","last_active":"2024-09-13T18:35:18.986335+0000","last_peered":"2024-09-13T18:35:18.986335+0000","last_clean":"2024-09-13T18:35:18.986335+0000","last_became_active":"2024-09-13T18:35:11.730015+0000","last_became_peered":"2024-09-13T18:35:11.730015+0000","last_unstale":"2024-09-13T18:35:18.986335+0000","last_undegraded":"2024-09-13T18:35:18.986335+0000","last_fullsized":"2024-09-13T18:35:18.986335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:43.650449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985381+0000","last_change":"2024-09-13T18:35:13.735587+0000","last_active":"2024-09-13T18:35:18.985381+0000","last_peered":"2024-09-13T18:35:18.985381+0000","last_clean":"2024-09-13T18:35:18.985381+0000","last_became_active":"2024-09-13T18:35:11.733034+0000","last_became_peered":"2024-09-13T18:35:11.733034+0000","last_unstale":"2024-09-13T18:35:18.985381+0000","last_undegraded":"2024-09-13T18:35:18.985381+0000","last_fullsized":"2024-09-13T18:35:18.985381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:01:22.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:19.347949+0000","last_change":"2024-09-13T18:35:14.461158+0000","last_active":"2024-09-13T18:35:19.347949+0000","last_peered":"2024-09-13T18:35:19.347949+0000","last_clean":"2024-09-13T18:35:19.347949+0000","last_became_active":"2024-09-13T18:35:11.729626+0000","last_became_peered":"2024-09-13T18:35:11.729626+0000","last_unstale":"2024-09-13T18:35:19.347949+0000","last_undegraded":"2024-09-13T18:35:19.347949+0000","last_fullsized":"2024-09-13T18:35:19.347949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:10.111028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985349+0000","last_change":"2024-09-13T18:35:13.735139+0000","last_active":"2024-09-13T18:35:18.985349+0000","last_peered":"2024-09-13T18:35:18.985349+0000","last_clean":"2024-09-13T18:35:18.985349+0000","last_became_active":"2024-09-13T18:35:11.733025+0000","last_became_peered":"2024-09-13T18:35:11.733025+0000","last_unstale":"2024-09-13T18:35:18.985349+0000","last_undegraded":"2024-09-13T18:35:18.985349+0000","last_fullsized":"2024-09-13T18:35:18.985349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.647913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.985651+0000","last_change":"2024-09-13T18:35:13.735186+0000","last_active":"2024-09-13T18:35:18.985651+0000","last_peered":"2024-09-13T18:35:18.985651+0000","last_clean":"2024-09-13T18:35:18.985651+0000","last_became_active":"2024-09-13T18:35:11.727196+0000","last_became_peered":"2024-09-13T18:35:11.727196+0000","last_unstale":"2024-09-13T18:35:18.985651+0000","last_undegraded":"2024-09-13T18:35:18.985651+0000","last_fullsized":"2024-09-13T18:35:18.985651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:18.070546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":82,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T18:35:18.986588+0000","last_change":"2024-09-13T18:35:07.177729+0000","last_active":"2024-09-13T18:35:18.986588+0000","last_peered":"2024-09-13T18:35:18.986588+0000","last_clean":"2024-09-13T18:35:18.986588+0000","last_became_active":"2024-09-13T18:35:07.176786+0000","last_became_peered":"2024-09-13T18:35:07.176786+0000","last_unstale":"2024-09-13T18:35:18.986588+0000","last_undegraded":"2024-09-13T18:35:18.986588+0000","last_fullsized":"2024-09-13T18:35:18.986588+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:02.631221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":50564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.441},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.501},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"last":0.47099999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":172032,"data_stored":50545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.438}]},{"osd":3,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.495},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004}]}]},{"osd":1,"up_from":9,"seq":38654705683,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":770048,"data_stored":640932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.309},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.27100000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29699999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705684,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":761856,"data_stored":640302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:26.020 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:36:26.020 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T18:36:26.020 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T18:36:26.020 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T18:36:26.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.112+0000 7fe472983640 1 Processor -- start 2024-09-13T18:36:26.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.112+0000 7fe472983640 1 -- start start 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe472983640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 0x7fe46c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe472983640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe46c06bd60 0x7fe46c12d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe472983640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 0x7fe46c12de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe472983640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c06d900 con 0x7fe46c06bd60 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe472983640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c12e3d0 con 0x7fe46c12da20 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe472983640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c06c4a0 con 0x7fe46c06b440 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 0x7fe46c12de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 0x7fe46c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 0x7fe46c12de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35462/0 (socket says 172.21.15.177:35462) 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 0x7fe46c06b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:33076/0 (socket says 172.21.15.177:33076) 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 -- 172.21.15.177:0/1805495500 learned_addr learned my addr 172.21.15.177:0/1805495500 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe46b7fe640 1 --2- 172.21.15.177:0/1805495500 >> v2:172.21.15.177:3300/0 conn(0x7fe46c06bd60 0x7fe46c12d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 -- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 msgr2=0x7fe46c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:26.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 --2- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 0x7fe46c06b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 -- 172.21.15.177:0/1805495500 >> v2:172.21.15.177:3300/0 conn(0x7fe46c06bd60 msgr2=0x7fe46c12d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:26.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 --2- 172.21.15.177:0/1805495500 >> v2:172.21.15.177:3300/0 conn(0x7fe46c06bd60 0x7fe46c12d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe470f20640 1 -- 172.21.15.177:0/1805495500 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c06db60 con 0x7fe46c12da20 2024-09-13T18:36:26.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe46bfff640 1 --2- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 0x7fe46c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.113+0000 7fe46b7fe640 1 --2- 172.21.15.177:0/1805495500 >> v2:172.21.15.177:3300/0 conn(0x7fe46c06bd60 0x7fe46c12d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.114+0000 7fe470f20640 1 --2- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 0x7fe46c12de90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe464009e50 tx=0x7fe46402a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1099aab66ec70eef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.114+0000 7fe46affd640 1 -- 172.21.15.177:0/1805495500 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464031600 con 0x7fe46c12da20 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.114+0000 7fe46affd640 1 -- 172.21.15.177:0/1805495500 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46402d050 con 0x7fe46c12da20 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.114+0000 7fe46affd640 1 -- 172.21.15.177:0/1805495500 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464031980 con 0x7fe46c12da20 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 -- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 msgr2=0x7fe46c12de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 --2- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 0x7fe46c12de90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe464009e50 tx=0x7fe46402a560 comp rx=0 tx=0).stop 2024-09-13T18:36:26.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 -- 172.21.15.177:0/1805495500 shutdown_connections 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 --2- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c12da20 0x7fe46c12de90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 --2- 172.21.15.177:0/1805495500 >> v2:172.21.15.177:3300/0 conn(0x7fe46c06bd60 0x7fe46c12d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 --2- 172.21.15.177:0/1805495500 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06b440 0x7fe46c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 -- 172.21.15.177:0/1805495500 >> 172.21.15.177:0/1805495500 conn(0x7fe46c09fe10 msgr2=0x7fe46c0a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 -- 172.21.15.177:0/1805495500 shutdown_connections 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 -- 172.21.15.177:0/1805495500 wait complete. 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.115+0000 7fe472983640 1 Processor -- start 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 -- start start 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 0x7fe46c1c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06bd60 0x7fe46c1c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe46c12da20 0x7fe46c1c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c127f00 con 0x7fe46c12da20 2024-09-13T18:36:26.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1310e0 con 0x7fe46c06b440 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe472983640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c130e00 con 0x7fe46c06bd60 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 0x7fe46c1c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 0x7fe46c1c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35466/0 (socket says 172.21.15.177:35466) 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe470f20640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe46c12da20 0x7fe46c1c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 -- 172.21.15.177:0/1524596059 learned_addr learned my addr 172.21.15.177:0/1524596059 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46b7fe640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06bd60 0x7fe46c1c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 -- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06bd60 msgr2=0x7fe46c1c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06bd60 0x7fe46c1c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 -- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe46c12da20 msgr2=0x7fe46c1c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe46c12da20 0x7fe46c1c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46bfff640 1 -- 172.21.15.177:0/1524596059 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c009400 con 0x7fe46c06b440 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe470f20640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe46c12da20 0x7fe46c1c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.116+0000 7fe46b7fe640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06bd60 0x7fe46c1c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.117+0000 7fe46bfff640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 0x7fe46c1c28e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00ec10 tx=0x7fe45c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.117+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c009760 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.117+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c00cc30 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.117+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe454009400 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.117+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c013470 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.117+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe464009b30 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.118+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fe45c0135d0 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.118+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c12d4e0 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.118+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fe45c05ced0 con 0x7fe46c06b440 2024-09-13T18:36:26.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.122+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fe45c03ab60 con 0x7fe46c06b440 2024-09-13T18:36:26.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.277+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe46c12f7f0 con 0x7fe46c06b440 2024-09-13T18:36:26.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.277+0000 7fe468ff9640 1 -- 172.21.15.177:0/1524596059 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe45c017070 con 0x7fe46c06b440 2024-09-13T18:36:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:26.281 INFO:teuthology.orchestra.run.smithi177.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T18:36:26.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 >> v1:172.21.15.177:6816/516212916 conn(0x7fe44c0474d0 legacy=0x7fe44c049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 msgr2=0x7fe46c1c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 0x7fe46c1c28e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00ec10 tx=0x7fe45c00c520 comp rx=0 tx=0).stop 2024-09-13T18:36:26.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 shutdown_connections 2024-09-13T18:36:26.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe46c12da20 0x7fe46c1c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe46c06bd60 0x7fe46c1c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 --2- 172.21.15.177:0/1524596059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe46c06b440 0x7fe46c1c28e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 >> 172.21.15.177:0/1524596059 conn(0x7fe46c09fe10 msgr2=0x7fe46c09bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 shutdown_connections 2024-09-13T18:36:26.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.279+0000 7fe472983640 1 -- 172.21.15.177:0/1524596059 wait complete. 2024-09-13T18:36:26.293 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T18:36:26.293 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-09-13T18:36:26.304 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T18:36:26.304 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:26.338 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T18:36:26.338 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-13T18:36:26.476 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.472+0000 7f5afb39d640 1 Processor -- start 2024-09-13T18:36:26.476 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afb39d640 1 -- start start 2024-09-13T18:36:26.477 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afb39d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5af406ca30 0x7f5af4132bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.477 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afb39d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ce60 0x7f5af4135110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afb39d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af406d6b0 con 0x7f5af4099430 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afb39d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af406d920 con 0x7f5af406ca30 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afb39d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4099ee0 con 0x7f5af406ce60 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afab9c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ce60 0x7f5af4135110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5af9b9a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5af406ca30 0x7f5af4132bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afa39b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5af4099430 0x7f5af406c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58220/0 (socket says 172.21.15.177:58220) 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.473+0000 7f5afa39b640 1 -- 172.21.15.177:0/2784905044 learned_addr learned my addr 172.21.15.177:0/2784905044 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9b9a640 1 -- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ce60 msgr2=0x7f5af4135110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9b9a640 1 --2- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ce60 0x7f5af4135110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9b9a640 1 -- 172.21.15.177:0/2784905044 >> v1:172.21.15.177:6789/0 conn(0x7f5af4099430 legacy=0x7f5af406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9b9a640 1 -- 172.21.15.177:0/2784905044 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4136660 con 0x7f5af406ca30 2024-09-13T18:36:26.478 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5afab9c640 1 --2- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ce60 0x7f5af4135110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.479 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9b9a640 1 --2- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af406ca30 0x7f5af4132bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80097a0 tx=0x7f5ae802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed8570d5f16dcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.479 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9399640 1 -- 172.21.15.177:0/2784905044 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae802adb0 con 0x7f5af406ca30 2024-09-13T18:36:26.479 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9399640 1 -- 172.21.15.177:0/2784905044 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8032440 con 0x7f5af406ca30 2024-09-13T18:36:26.479 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.474+0000 7f5af9399640 1 -- 172.21.15.177:0/2784905044 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80327d0 con 0x7f5af406ca30 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 -- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af406ca30 msgr2=0x7f5af4132bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 --2- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af406ca30 0x7f5af4132bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80097a0 tx=0x7f5ae802a060 comp rx=0 tx=0).stop 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 -- 172.21.15.177:0/2784905044 shutdown_connections 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 --2- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af406ca30 0x7f5af4132bc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 --2- 172.21.15.177:0/2784905044 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ce60 0x7f5af4135110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 -- 172.21.15.177:0/2784905044 >> 172.21.15.177:0/2784905044 conn(0x7f5af409a520 msgr2=0x7f5af409a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 -- 172.21.15.177:0/2784905044 shutdown_connections 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.475+0000 7f5afb39d640 1 -- 172.21.15.177:0/2784905044 wait complete. 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 Processor -- start 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 -- start start 2024-09-13T18:36:26.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ca30 0x7f5af408f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af406ce60 0x7f5af408f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 0x7f5af408fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4090420 con 0x7f5af4099430 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4134000 con 0x7f5af406ce60 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afb39d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40710f0 con 0x7f5af406ca30 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afa39b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ca30 0x7f5af408f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afab9c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 0x7f5af408fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5af9b9a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af406ce60 0x7f5af408f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afab9c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 0x7f5af408fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47658/0 (socket says 172.21.15.177:47658) 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afa39b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ca30 0x7f5af408f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:33094/0 (socket says 172.21.15.177:33094) 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.476+0000 7f5afab9c640 1 -- 172.21.15.177:0/1069747429 learned_addr learned my addr 172.21.15.177:0/1069747429 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afab9c640 1 -- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ca30 msgr2=0x7f5af408f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afab9c640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ca30 0x7f5af408f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afab9c640 1 -- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af406ce60 msgr2=0x7f5af408f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afab9c640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af406ce60 0x7f5af408f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afab9c640 1 -- 172.21.15.177:0/1069747429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009a70 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afab9c640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 0x7f5af408fdf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00efc0 tx=0x7f5aec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec009cc0 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00f040 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec003990 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8009400 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.477+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af4090b40 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.478+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f5aec014480 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.478+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af40917b0 con 0x7f5af4099430 2024-09-13T18:36:26.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.478+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f5aec05d430 con 0x7f5af4099430 2024-09-13T18:36:26.485 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.482+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f5aec03b1e0 con 0x7f5af4099430 2024-09-13T18:36:26.601 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.598+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5aec05c0a0 con 0x7f5af4099430 2024-09-13T18:36:26.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.634+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7f5af41360a0 con 0x7f5af4099430 2024-09-13T18:36:26.638 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-09-13T18:36:26.635+0000 7f0452cd1640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-13T18:36:26.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.635+0000 7f5ae6ffd640 1 -- 172.21.15.177:0/1069747429 <== mon.0 v2:172.21.15.177:3300/0 8 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 osd.0 is already down. v21) v1 ==== 93+0+0 (secure 0 0 0) 0x7f5aec018020 con 0x7f5af4099430 2024-09-13T18:36:26.639 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0 is already down. 2024-09-13T18:36:26.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 >> v1:172.21.15.177:6816/516212916 conn(0x7f5ad80474d0 legacy=0x7f5ad8049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 msgr2=0x7f5af408fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 0x7f5af408fdf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00efc0 tx=0x7f5aec00c490 comp rx=0 tx=0).stop 2024-09-13T18:36:26.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 shutdown_connections 2024-09-13T18:36:26.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af4099430 0x7f5af408fdf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af406ce60 0x7f5af408f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 --2- 172.21.15.177:0/1069747429 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5af406ca30 0x7f5af408f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 >> 172.21.15.177:0/1069747429 conn(0x7f5af409a520 msgr2=0x7f5af412cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 shutdown_connections 2024-09-13T18:36:26.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.637+0000 7f5afb39d640 1 -- 172.21.15.177:0/1069747429 wait complete. 2024-09-13T18:36:26.651 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T18:36:26.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T18:36:26.654 INFO:tasks.ceph.osd.0:Started 2024-09-13T18:36:26.654 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T18:36:26.654 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:26.686 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T18:36:26.686 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.821+0000 7f277065a640 1 Processor -- start 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.821+0000 7f277065a640 1 -- start start 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f277065a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2768095270 0x7f2768095650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f277065a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f276806cca0 0x7f2768095b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f277065a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 0x7f276806b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f277065a640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276806db30 con 0x7f276806cca0 2024-09-13T18:36:26.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f277065a640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276806e080 con 0x7f276806b1d0 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f277065a640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f276806e390 con 0x7f2768095270 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276ebf7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 0x7f276806b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276e3f6640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2768095270 0x7f2768095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276ebf7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 0x7f276806b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:35484/0 (socket says 172.21.15.177:35484) 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276dbf5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f276806cca0 0x7f2768095b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276ebf7640 1 -- 172.21.15.177:0/3336921987 learned_addr learned my addr 172.21.15.177:0/3336921987 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276e3f6640 1 -- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 msgr2=0x7f276806b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276e3f6640 1 --2- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 0x7f276806b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276e3f6640 1 -- 172.21.15.177:0/3336921987 >> v2:172.21.15.177:3300/0 conn(0x7f276806cca0 msgr2=0x7f2768095b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276e3f6640 1 --2- 172.21.15.177:0/3336921987 >> v2:172.21.15.177:3300/0 conn(0x7f276806cca0 0x7f2768095b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276e3f6640 1 -- 172.21.15.177:0/3336921987 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768097b90 con 0x7f2768095270 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276ebf7640 1 --2- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 0x7f276806b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.822+0000 7f276dbf5640 1 --2- 172.21.15.177:0/3336921987 >> v2:172.21.15.177:3300/0 conn(0x7f276806cca0 0x7f2768095b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.823+0000 7f276e3f6640 1 --2- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2768095270 0x7f2768095650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27640097a0 tx=0x7f2764029fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ceb40865e073259 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.823+0000 7f276d3f4640 1 -- 172.21.15.177:0/3336921987 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276402ad20 con 0x7f2768095270 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.823+0000 7f276d3f4640 1 -- 172.21.15.177:0/3336921987 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276402ae80 con 0x7f2768095270 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.823+0000 7f276d3f4640 1 -- 172.21.15.177:0/3336921987 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764032660 con 0x7f2768095270 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.823+0000 7f277065a640 1 -- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2768095270 msgr2=0x7f2768095650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.823+0000 7f277065a640 1 --2- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2768095270 0x7f2768095650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27640097a0 tx=0x7f2764029fd0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 -- 172.21.15.177:0/3336921987 shutdown_connections 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 --2- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806b1d0 0x7f276806b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 --2- 172.21.15.177:0/3336921987 >> v2:172.21.15.177:3300/0 conn(0x7f276806cca0 0x7f2768095b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 --2- 172.21.15.177:0/3336921987 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2768095270 0x7f2768095650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 -- 172.21.15.177:0/3336921987 >> 172.21.15.177:0/3336921987 conn(0x7f2768098ba0 msgr2=0x7f2768096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 -- 172.21.15.177:0/3336921987 shutdown_connections 2024-09-13T18:36:26.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 -- 172.21.15.177:0/3336921987 wait complete. 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 Processor -- start 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.824+0000 7f277065a640 1 -- start start 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f277065a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 0x7f2768128e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f277065a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806cca0 0x7f2768129340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f277065a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2768095270 0x7f2768129880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f277065a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276812c5f0 con 0x7f2768095270 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f277065a640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276812c900 con 0x7f276806cca0 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f277065a640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f276812b4a0 con 0x7f276806b1d0 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276e3f6640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 0x7f2768128e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806cca0 0x7f2768129340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276ebf7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2768095270 0x7f2768129880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276e3f6640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 0x7f2768128e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:33132/0 (socket says 172.21.15.177:33132) 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276e3f6640 1 -- 172.21.15.177:0/3946190669 learned_addr learned my addr 172.21.15.177:0/3946190669 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 -- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 msgr2=0x7f2768128e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 0x7f2768128e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 -- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2768095270 msgr2=0x7f2768129880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2768095270 0x7f2768129880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 -- 172.21.15.177:0/3946190669 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760009b30 con 0x7f276806cca0 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276e3f6640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 0x7f2768128e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276ebf7640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2768095270 0x7f2768129880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.825+0000 7f276dbf5640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806cca0 0x7f2768129340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f275800e900 tx=0x7f275800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.826+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275800ca00 con 0x7f276806cca0 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.826+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275800cb60 con 0x7f276806cca0 2024-09-13T18:36:26.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.826+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2758009400 con 0x7f276806cca0 2024-09-13T18:36:26.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.826+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758013470 con 0x7f276806cca0 2024-09-13T18:36:26.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.826+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2764009400 con 0x7f276806cca0 2024-09-13T18:36:26.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.827+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276806b640 con 0x7f276806cca0 2024-09-13T18:36:26.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.827+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f27580135d0 con 0x7f276806cca0 2024-09-13T18:36:26.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.827+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f275805cb10 con 0x7f276806cca0 2024-09-13T18:36:26.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.830+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f275803a850 con 0x7f276806cca0 2024-09-13T18:36:26.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:26.977+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7f2768073cc0 con 0x7f276806cca0 2024-09-13T18:36:26.980 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-09-13T18:36:26.977+0000 7f0452cd1640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-13T18:36:27.238 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:36:27.235+0000 7f0b07811640 -1 Falling back to public interface 2024-09-13T18:36:27.604 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.601+0000 7f275effd640 1 -- 172.21.15.177:0/3946190669 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v22) v1 ==== 89+0+0 (secure 0 0 0) 0x7f275803de70 con 0x7f276806cca0 2024-09-13T18:36:27.604 INFO:teuthology.orchestra.run.smithi177.stderr:marked down osd.1. 2024-09-13T18:36:27.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 >> v1:172.21.15.177:6816/516212916 conn(0x7f274c047560 legacy=0x7f274c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:27.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806cca0 msgr2=0x7f2768129340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:27.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806cca0 0x7f2768129340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f275800e900 tx=0x7f275800edc0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 shutdown_connections 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2768095270 0x7f2768129880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276806cca0 0x7f2768129340 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 --2- 172.21.15.177:0/3946190669 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f276806b1d0 0x7f2768128e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 >> 172.21.15.177:0/3946190669 conn(0x7f2768098ba0 msgr2=0x7f2768099950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 shutdown_connections 2024-09-13T18:36:27.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.603+0000 7f277065a640 1 -- 172.21.15.177:0/3946190669 wait complete. 2024-09-13T18:36:27.617 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T18:36:27.618 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T18:36:27.620 INFO:tasks.ceph.osd.1:Started 2024-09-13T18:36:27.620 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T18:36:27.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:27.654 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T18:36:27.654 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 Processor -- start 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 -- start start 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912406b440 0x7f912406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912412d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912406c7d0 con 0x7f912412d920 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912406cf60 con 0x7f912406bd60 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f912959d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9124070ee0 con 0x7f912406b440 2024-09-13T18:36:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f911bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912412d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9128d9c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f912412d920 0x7f9124134710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58250/0 (socket says 172.21.15.177:58250) 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9123fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912406b440 0x7f912406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9128d9c640 1 -- 172.21.15.177:0/504130344 learned_addr learned my addr 172.21.15.177:0/504130344 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f91237fe640 1 -- 172.21.15.177:0/504130344 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733423636 0 0) 0x7f912406c7d0 con 0x7f912412d920 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9123fff640 1 -- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 msgr2=0x7f912412d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9123fff640 1 --2- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912412d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9123fff640 1 -- 172.21.15.177:0/504130344 >> v1:172.21.15.177:6789/0 conn(0x7f912412d920 legacy=0x7f9124134710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f9123fff640 1 -- 172.21.15.177:0/504130344 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912406c280 con 0x7f912406b440 2024-09-13T18:36:27.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.790+0000 7f911bfff640 1 --2- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912412d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:27.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.791+0000 7f9123fff640 1 --2- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912406b440 0x7f912406b820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91140097d0 tx=0x7f911402a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e44076d8f78d717 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:27.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.791+0000 7f91237fe640 1 -- 172.21.15.177:0/504130344 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911402ad90 con 0x7f912406b440 2024-09-13T18:36:27.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.791+0000 7f91237fe640 1 -- 172.21.15.177:0/504130344 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114032440 con 0x7f912406b440 2024-09-13T18:36:27.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.791+0000 7f91237fe640 1 -- 172.21.15.177:0/504130344 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140327d0 con 0x7f912406b440 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 -- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912406b440 msgr2=0x7f912406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 --2- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912406b440 0x7f912406b820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91140097d0 tx=0x7f911402a040 comp rx=0 tx=0).stop 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 -- 172.21.15.177:0/504130344 shutdown_connections 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 --2- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912412d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 --2- 172.21.15.177:0/504130344 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912406b440 0x7f912406b820 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 -- 172.21.15.177:0/504130344 >> 172.21.15.177:0/504130344 conn(0x7f912409a630 msgr2=0x7f912409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 -- 172.21.15.177:0/504130344 shutdown_connections 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.792+0000 7f912959d640 1 -- 172.21.15.177:0/504130344 wait complete. 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 Processor -- start 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 -- start start 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 0x7f912408f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:27.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912408f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912412d920 0x7f912408fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124092b70 con 0x7f912406b440 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9124127cb0 con 0x7f912406bd60 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f912959d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91240926c0 con 0x7f912412d920 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f9123fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 0x7f912408f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f9128d9c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912412d920 0x7f912408fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f911bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912408f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f9123fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 0x7f912408f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:47686/0 (socket says 172.21.15.177:47686) 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.793+0000 7f9123fff640 1 -- 172.21.15.177:0/2686088906 learned_addr learned my addr 172.21.15.177:0/2686088906 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f9123fff640 1 -- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912412d920 msgr2=0x7f912408fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f9123fff640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912412d920 0x7f912408fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f9123fff640 1 -- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 msgr2=0x7f912408f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:27.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f9123fff640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912408f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f9123fff640 1 -- 172.21.15.177:0/2686088906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c009400 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f911bfff640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912408f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f9123fff640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 0x7f912408f370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f911402a560 tx=0x7f91140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911403d650 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911403d7d0 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9114009400 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911403db80 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.794+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91241c22f0 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.795+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9114039040 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.795+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9124067400 con 0x7f912406b440 2024-09-13T18:36:27.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.795+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f911407d110 con 0x7f912406b440 2024-09-13T18:36:27.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.799+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f9114007cc0 con 0x7f912406b440 2024-09-13T18:36:27.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:27.942+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7f912406cd10 con 0x7f912406b440 2024-09-13T18:36:27.945 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-09-13T18:36:27.942+0000 7f0452cd1640 -1 mon.a@0(leader).osd e22 definitely_dead 0 2024-09-13T18:36:28.205 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:36:28.202+0000 7f0a3580b640 -1 Falling back to public interface 2024-09-13T18:36:28.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.604+0000 7f91217fa640 1 -- 172.21.15.177:0/2686088906 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v23) v1 ==== 89+0+0 (secure 0 0 0) 0x7f911405e3b0 con 0x7f912406b440 2024-09-13T18:36:28.608 INFO:teuthology.orchestra.run.smithi177.stderr:marked down osd.2. 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 >> v1:172.21.15.177:6816/516212916 conn(0x7f9104047560 legacy=0x7f9104049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 msgr2=0x7f912408f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 0x7f912408f370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f911402a560 tx=0x7f91140046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 shutdown_connections 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912412d920 0x7f912408fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912406bd60 0x7f912408f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 --2- 172.21.15.177:0/2686088906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912406b440 0x7f912408f370 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.606+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 >> 172.21.15.177:0/2686088906 conn(0x7f912409a630 msgr2=0x7f912409e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:28.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.607+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 shutdown_connections 2024-09-13T18:36:28.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.607+0000 7f912959d640 1 -- 172.21.15.177:0/2686088906 wait complete. 2024-09-13T18:36:28.621 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T18:36:28.621 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T18:36:28.624 INFO:tasks.ceph.osd.2:Started 2024-09-13T18:36:28.624 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T18:36:28.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:36:28.654 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T18:36:28.654 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2024-09-13T18:36:28.793 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.788+0000 7fdaced83640 1 Processor -- start 2024-09-13T18:36:28.793 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.788+0000 7fdaced83640 1 -- start start 2024-09-13T18:36:28.793 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaced83640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.793 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaced83640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.793 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaced83640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdac812da20 0x7fdac812dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.793 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaced83640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac806d840 con 0x7fdac812da20 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaced83640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac806c4a0 con 0x7fdac806b440 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaced83640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdac806db60 con 0x7fdac806bd60 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdac812da20 0x7fdac812dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaccb1f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdac812da20 0x7fdac812dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59738/0 (socket says 172.21.15.177:59738) 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaccb1f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac806b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54194/0 (socket says 172.21.15.177:54194) 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:35958/0 (socket says 172.21.15.177:35958) 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 -- 172.21.15.177:0/3104295722 learned_addr learned my addr 172.21.15.177:0/3104295722 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 -- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 msgr2=0x7fdac812d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 -- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 msgr2=0x7fdac806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:28.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac806b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdacd320640 1 -- 172.21.15.177:0/3104295722 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac806c140 con 0x7fdac812da20 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdac7fff640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.789+0000 7fdaccb1f640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac806b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.790+0000 7fdacd320640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812dea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009e50 tx=0x7fdac002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996532399ff69ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.790+0000 7fdac77fe640 1 -- 172.21.15.177:0/3104295722 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0031600 con 0x7fdac812da20 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.790+0000 7fdac77fe640 1 -- 172.21.15.177:0/3104295722 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac002d050 con 0x7fdac812da20 2024-09-13T18:36:28.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.790+0000 7fdac77fe640 1 -- 172.21.15.177:0/3104295722 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0031980 con 0x7fdac812da20 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 -- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 msgr2=0x7fdac812dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812dea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009e50 tx=0x7fdac002a560 comp rx=0 tx=0).stop 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 -- 172.21.15.177:0/3104295722 shutdown_connections 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812dea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 --2- 172.21.15.177:0/3104295722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac806b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 -- 172.21.15.177:0/3104295722 >> 172.21.15.177:0/3104295722 conn(0x7fdac809fe10 msgr2=0x7fdac80a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 -- 172.21.15.177:0/3104295722 shutdown_connections 2024-09-13T18:36:28.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.791+0000 7fdaced83640 1 -- 172.21.15.177:0/3104295722 wait complete. 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 Processor -- start 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 -- start start 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac812e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8127f00 con 0x7fdac812da20 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac812f8e0 con 0x7fdac806b440 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaced83640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8130cd0 con 0x7fdac806bd60 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdaccb1f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac812e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdacd320640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdacd320640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59748/0 (socket says 172.21.15.177:59748) 2024-09-13T18:36:28.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.792+0000 7fdacd320640 1 -- 172.21.15.177:0/4077673637 learned_addr learned my addr 172.21.15.177:0/4077673637 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaccb1f640 1 -- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 msgr2=0x7fdac812eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaccb1f640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaccb1f640 1 -- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 msgr2=0x7fdac812f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaccb1f640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaccb1f640 1 -- 172.21.15.177:0/4077673637 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009400 con 0x7fdac806b440 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaccb1f640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac812e5d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdab800e870 tx=0x7fdab800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800c940 con 0x7fdac806b440 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab800caa0 con 0x7fdac806b440 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800ce30 con 0x7fdac806b440 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc009400 con 0x7fdac806b440 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.793+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac0009b30 con 0x7fdac806b440 2024-09-13T18:36:28.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.794+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fdab8013470 con 0x7fdac806b440 2024-09-13T18:36:28.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.794+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8070b50 con 0x7fdac806b440 2024-09-13T18:36:28.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.794+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fdab805cb40 con 0x7fdac806b440 2024-09-13T18:36:28.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.798+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fdab803a760 con 0x7fdac806b440 2024-09-13T18:36:28.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:28.941+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7fdac812d4e0 con 0x7fdac806b440 2024-09-13T18:36:28.945 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-09-13T18:36:28.942+0000 7f0452cd1640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2024-09-13T18:36:29.212 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:36:29.209+0000 7fdf453bb640 -1 Falling back to public interface 2024-09-13T18:36:29.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.609+0000 7fdac57fa640 1 -- 172.21.15.177:0/4077673637 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v24) v1 ==== 89+0+0 (secure 0 0 0) 0x7fdab8021070 con 0x7fdac806b440 2024-09-13T18:36:29.612 INFO:teuthology.orchestra.run.smithi177.stderr:marked down osd.3. 2024-09-13T18:36:29.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 >> v1:172.21.15.177:6816/516212916 conn(0x7fdab4047560 legacy=0x7fdab4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 msgr2=0x7fdac812e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac812e5d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdab800e870 tx=0x7fdab800ed30 comp rx=0 tx=0).stop 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 shutdown_connections 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac812da20 0x7fdac812f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fdac806bd60 0x7fdac812eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 --2- 172.21.15.177:0/4077673637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdac806b440 0x7fdac812e5d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 >> 172.21.15.177:0/4077673637 conn(0x7fdac809fe10 msgr2=0x7fdac809bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 shutdown_connections 2024-09-13T18:36:29.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.611+0000 7fdaced83640 1 -- 172.21.15.177:0/4077673637 wait complete. 2024-09-13T18:36:29.627 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T18:36:29.628 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T18:36:29.630 INFO:tasks.ceph.osd.3:Started 2024-09-13T18:36:29.630 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T18:36:29.630 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:29.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 Processor -- start 2024-09-13T18:36:29.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 -- start start 2024-09-13T18:36:29.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875806b440 0x7f875806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:29.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 0x7f875812d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875806c4a0 con 0x7f875812da20 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875806d900 con 0x7f875806b440 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875eab1640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f875806db60 con 0x7f875806bd60 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f8757fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 0x7f875812d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875c84d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875806b440 0x7f875806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f8757fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 0x7f875812d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:35990/0 (socket says 172.21.15.177:35990) 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f875d04e640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f875812da20 0x7f875812dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:49266/0 (socket says 172.21.15.177:49266) 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.725+0000 7f8757fff640 1 -- 172.21.15.177:0/3341706077 learned_addr learned my addr 172.21.15.177:0/3341706077 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f8757fff640 1 -- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875806b440 msgr2=0x7f875806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f8757fff640 1 --2- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875806b440 0x7f875806b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f87577fe640 1 -- 172.21.15.177:0/3341706077 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365967252 0 0) 0x7f875806c4a0 con 0x7f875812da20 2024-09-13T18:36:29.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f8757fff640 1 -- 172.21.15.177:0/3341706077 >> v1:172.21.15.177:6789/0 conn(0x7f875812da20 legacy=0x7f875812dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f8757fff640 1 -- 172.21.15.177:0/3341706077 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875806c140 con 0x7f875806bd60 2024-09-13T18:36:29.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f875c84d640 1 --2- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875806b440 0x7f875806b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:29.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.726+0000 7f8757fff640 1 --2- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 0x7f875812d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f87440098a0 tx=0x7f874402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62638ceee1e6759b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:29.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f87577fe640 1 -- 172.21.15.177:0/3341706077 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874402ae60 con 0x7f875806bd60 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f87577fe640 1 -- 172.21.15.177:0/3341706077 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8744032440 con 0x7f875806bd60 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f87577fe640 1 -- 172.21.15.177:0/3341706077 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440327d0 con 0x7f875806bd60 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 -- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 msgr2=0x7f875812d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 --2- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 0x7f875812d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f87440098a0 tx=0x7f874402a110 comp rx=0 tx=0).stop 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 -- 172.21.15.177:0/3341706077 shutdown_connections 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 --2- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806bd60 0x7f875812d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 --2- 172.21.15.177:0/3341706077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875806b440 0x7f875806b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 -- 172.21.15.177:0/3341706077 >> 172.21.15.177:0/3341706077 conn(0x7f875809fe10 msgr2=0x7f87580a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 -- 172.21.15.177:0/3341706077 shutdown_connections 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.727+0000 7f875eab1640 1 -- 172.21.15.177:0/3341706077 wait complete. 2024-09-13T18:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.728+0000 7f875eab1640 1 Processor -- start 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.728+0000 7f875eab1640 1 -- start start 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875eab1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806b440 0x7f875812e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875eab1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 0x7f875812eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875eab1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875812da20 0x7f875812f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875eab1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758135660 con 0x7f875806bd60 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875eab1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8758131060 con 0x7f875812da20 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875eab1640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8758131430 con 0x7f875806b440 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 0x7f875812eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:29.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875c84d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806b440 0x7f875812e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 0x7f875812eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59754/0 (socket says 172.21.15.177:59754) 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 -- 172.21.15.177:0/2925885998 learned_addr learned my addr 172.21.15.177:0/2925885998 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875d04e640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875812da20 0x7f875812f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 -- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806b440 msgr2=0x7f875812e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806b440 0x7f875812e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 -- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875812da20 msgr2=0x7f875812f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875812da20 0x7f875812f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 -- 172.21.15.177:0/2925885998 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c009400 con 0x7f875806bd60 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875c84d640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806b440 0x7f875812e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f875d04e640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875812da20 0x7f875812f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.729+0000 7f8757fff640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 0x7f875812eb10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8744035040 tx=0x7f87440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.730+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744030850 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.730+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87440309b0 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.730+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744030d60 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.730+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87440094d0 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.730+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8758130880 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.731+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f8744039040 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.731+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875806b820 con 0x7f875806bd60 2024-09-13T18:36:29.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.731+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f874407cd60 con 0x7f875806bd60 2024-09-13T18:36:29.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.735+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f874405e410 con 0x7f875806bd60 2024-09-13T18:36:29.877 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.874+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8758129820 con 0x7f875806bd60 2024-09-13T18:36:29.877 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.874+0000 7f87557fa640 1 -- 172.21.15.177:0/2925885998 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+8481 (secure 0 0 0) 0x7f874405e000 con 0x7f875806bd60 2024-09-13T18:36:29.877 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:29.878 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":24,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:36:29.602766+0000","last_up_change":"2024-09-13T18:36:29.602766+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":32756}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":32756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":32756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":32756}]},"public_addr":"172.21.15.177:6804/32756","cluster_addr":"172.21.15.177:6806/32756","heartbeat_back_addr":"172.21.15.177:6810/32756","heartbeat_front_addr":"172.21.15.177:6809/32756","state":["exists"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":22},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":23},{"osd":3,"down_stamp":"2024-09-13T18:36:29.602766+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:0/2408340411":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/2735606740":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/3625374812":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/1782828617":"2024-09-14T18:36:22.577532+0000","172.21.15.177:6816/32265":"2024-09-14T18:36:22.577532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:29.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 >> v1:172.21.15.177:6816/516212916 conn(0x7f873c047560 legacy=0x7f873c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 msgr2=0x7f875812eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:29.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 0x7f875812eb10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8744035040 tx=0x7f87440046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 shutdown_connections 2024-09-13T18:36:29.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f875812da20 0x7f875812f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f875806bd60 0x7f875812eb10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 --2- 172.21.15.177:0/2925885998 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f875806b440 0x7f875812e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:29.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 >> 172.21.15.177:0/2925885998 conn(0x7f875809fe10 msgr2=0x7f8758127490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:29.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 shutdown_connections 2024-09-13T18:36:29.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:29.876+0000 7f875eab1640 1 -- 172.21.15.177:0/2925885998 wait complete. 2024-09-13T18:36:30.215 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:36:30.211+0000 7f3cf362f640 -1 Falling back to public interface 2024-09-13T18:36:30.789 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:36:30.786+0000 7f0b07811640 -1 osd.0 20 log_to_monitors true 2024-09-13T18:36:31.633 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-09-13T18:36:31.630+0000 7f0afaefb640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:36:31.795 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:36:31.792+0000 7f0a3580b640 -1 osd.1 20 log_to_monitors true 2024-09-13T18:36:32.640 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-09-13T18:36:32.637+0000 7f0a28ea5640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:36:32.781 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:36:32.778+0000 7fdf453bb640 -1 osd.2 22 log_to_monitors true 2024-09-13T18:36:32.892 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:32.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 Processor -- start 2024-09-13T18:36:32.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 -- start start 2024-09-13T18:36:32.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812a370 0x7fa41812a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 0x7fa41812d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa418098be0 con 0x7fa41812db40 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41809a310 con 0x7fa41812a370 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41ed5e640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa418099400 con 0x7fa41812ac90 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa417fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 0x7fa41812d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa41cafa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812a370 0x7fa41812a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa417fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 0x7fa41812d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36006/0 (socket says 172.21.15.177:36006) 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.987+0000 7fa417fff640 1 -- 172.21.15.177:0/2484792683 learned_addr learned my addr 172.21.15.177:0/2484792683 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa41cafa640 1 -- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 msgr2=0x7fa41812d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa41cafa640 1 --2- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 0x7fa41812d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa4177fe640 1 -- 172.21.15.177:0/2484792683 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880232900 0 0) 0x7fa418098be0 con 0x7fa41812db40 2024-09-13T18:36:32.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa41cafa640 1 -- 172.21.15.177:0/2484792683 >> v1:172.21.15.177:6789/0 conn(0x7fa41812db40 legacy=0x7fa41812dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:32.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa41cafa640 1 -- 172.21.15.177:0/2484792683 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418099be0 con 0x7fa41812a370 2024-09-13T18:36:32.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa417fff640 1 --2- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 0x7fa41812d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:32.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.988+0000 7fa41cafa640 1 --2- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812a370 0x7fa41812a750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0097d0 tx=0x7fa40c02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf9106f9369d8d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:32.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa4177fe640 1 -- 172.21.15.177:0/2484792683 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c02ad90 con 0x7fa41812a370 2024-09-13T18:36:32.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa4177fe640 1 -- 172.21.15.177:0/2484792683 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c032440 con 0x7fa41812a370 2024-09-13T18:36:32.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa4177fe640 1 -- 172.21.15.177:0/2484792683 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0327d0 con 0x7fa41812a370 2024-09-13T18:36:32.995 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 -- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812a370 msgr2=0x7fa41812a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812a370 0x7fa41812a750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0097d0 tx=0x7fa40c02a040 comp rx=0 tx=0).stop 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 -- 172.21.15.177:0/2484792683 shutdown_connections 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812ac90 0x7fa41812d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/2484792683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812a370 0x7fa41812a750 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 -- 172.21.15.177:0/2484792683 >> 172.21.15.177:0/2484792683 conn(0x7fa4180a0160 msgr2=0x7fa4180a0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 -- 172.21.15.177:0/2484792683 shutdown_connections 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.989+0000 7fa41ed5e640 1 -- 172.21.15.177:0/2484792683 wait complete. 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.990+0000 7fa41ed5e640 1 Processor -- start 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.990+0000 7fa41ed5e640 1 -- start start 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.990+0000 7fa41ed5e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 0x7fa418125040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:32.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.990+0000 7fa41ed5e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 0x7fa418125580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41ed5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812db40 0x7fa418125ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41ed5e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418071350 con 0x7fa41812ac90 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41ed5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa418126000 con 0x7fa41812db40 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41ed5e640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa418127300 con 0x7fa41812a370 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 0x7fa418125580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41cafa640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 0x7fa418125040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 0x7fa418125580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59756/0 (socket says 172.21.15.177:59756) 2024-09-13T18:36:32.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41cafa640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 0x7fa418125040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36014/0 (socket says 172.21.15.177:36014) 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 -- 172.21.15.177:0/4017244166 learned_addr learned my addr 172.21.15.177:0/4017244166 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41d2fb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812db40 0x7fa418125ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 -- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 msgr2=0x7fa418125040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 0x7fa418125040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 -- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812db40 msgr2=0x7fa418125ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812db40 0x7fa418125ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 -- 172.21.15.177:0/4017244166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040094d0 con 0x7fa41812ac90 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41d2fb640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812db40 0x7fa418125ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41cafa640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 0x7fa418125040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa417fff640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 0x7fa418125580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa40400e9b0 tx=0x7fa40400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400ca00 con 0x7fa41812ac90 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40400cb60 con 0x7fa41812ac90 2024-09-13T18:36:32.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404013470 con 0x7fa41812ac90 2024-09-13T18:36:32.999 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c009400 con 0x7fa41812ac90 2024-09-13T18:36:32.999 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.991+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa418092710 con 0x7fa41812ac90 2024-09-13T18:36:32.999 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.992+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa418126f80 con 0x7fa41812ac90 2024-09-13T18:36:32.999 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.992+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa4040135d0 con 0x7fa41812ac90 2024-09-13T18:36:32.999 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.993+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fa40405cba0 con 0x7fa41812ac90 2024-09-13T18:36:32.999 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:32.996+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fa40403a7c0 con 0x7fa41812ac90 2024-09-13T18:36:33.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.140+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa41812a750 con 0x7fa41812ac90 2024-09-13T18:36:33.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.140+0000 7fa4157fa640 1 -- 172.21.15.177:0/4017244166 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+8929 (secure 0 0 0) 0x7fa40403dde0 con 0x7fa41812ac90 2024-09-13T18:36:33.144 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:33.144 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":26,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:36:32.625551+0000","last_up_change":"2024-09-13T18:36:32.625551+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.0), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":0,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":1669150124}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":1669150124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":1669150124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":1669150124}]},"public_addr":"172.21.15.177:6804/1669150124","cluster_addr":"172.21.15.177:6806/1669150124","heartbeat_back_addr":"172.21.15.177:6810/1669150124","heartbeat_front_addr":"172.21.15.177:6809/1669150124","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":32757}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":32757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":32757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":32757}]},"public_addr":"172.21.15.177:6812/32757","cluster_addr":"172.21.15.177:6813/32757","heartbeat_back_addr":"172.21.15.177:6815/32757","heartbeat_front_addr":"172.21.15.177:6814/32757","state":["exists"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":32758}]},"public_addr":"172.21.15.177:6805/32758","cluster_addr":"172.21.15.177:6807/32758","heartbeat_back_addr":"172.21.15.177:6811/32758","heartbeat_front_addr":"172.21.15.177:6808/32758","state":["exists"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":32755}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":32755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":32755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":32755}]},"public_addr":"172.21.15.177:6800/32755","cluster_addr":"172.21.15.177:6801/32755","heartbeat_back_addr":"172.21.15.177:6803/32755","heartbeat_front_addr":"172.21.15.177:6802/32755","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":22},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":23},{"osd":3,"down_stamp":"2024-09-13T18:36:29.602766+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:0/2408340411":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/2735606740":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/3625374812":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/1782828617":"2024-09-14T18:36:22.577532+0000","172.21.15.177:6816/32265":"2024-09-14T18:36:22.577532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:33.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 >> v1:172.21.15.177:6816/516212916 conn(0x7fa3fc047560 legacy=0x7fa3fc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 msgr2=0x7fa418125580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 0x7fa418125580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa40400e9b0 tx=0x7fa40400ee70 comp rx=0 tx=0).stop 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 shutdown_connections 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa41812db40 0x7fa418125ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa41812ac90 0x7fa418125580 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 --2- 172.21.15.177:0/4017244166 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa41812a370 0x7fa418125040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.142+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 >> 172.21.15.177:0/4017244166 conn(0x7fa4180a0160 msgr2=0x7fa41809a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.143+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 shutdown_connections 2024-09-13T18:36:33.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:33.143+0000 7fa41ed5e640 1 -- 172.21.15.177:0/4017244166 wait complete. 2024-09-13T18:36:33.640 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-09-13T18:36:33.637+0000 7fdf39158640 -1 osd.2 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:36:33.811 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:36:33.808+0000 7f3cf362f640 -1 osd.3 23 log_to_monitors true 2024-09-13T18:36:34.648 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-09-13T18:36:34.645+0000 7f3ce73cc640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:36:36.157 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:36.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.263+0000 7fed64a72640 1 Processor -- start 2024-09-13T18:36:36.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.263+0000 7fed64a72640 1 -- start start 2024-09-13T18:36:36.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed64a72640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed60132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.267 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed64a72640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed60132d40 0x7fed60135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed64a72640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6006d840 con 0x7fed6006b440 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed64a72640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6006c4a0 con 0x7fed6006bf30 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed64a72640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed6006db60 con 0x7fed60132d40 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5ed9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed60132d40 0x7fed60135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5e59c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fed6006b440 0x7fed6006b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:49344/0 (socket says 172.21.15.177:49344) 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5dd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed60132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5e59c640 1 -- 172.21.15.177:0/1616755054 learned_addr learned my addr 172.21.15.177:0/1616755054 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5ed9d640 1 -- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 msgr2=0x7fed60132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5ed9d640 1 --2- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed60132800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5ed9d640 1 -- 172.21.15.177:0/1616755054 >> v1:172.21.15.177:6789/0 conn(0x7fed6006b440 legacy=0x7fed6006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5d59a640 1 -- 172.21.15.177:0/1616755054 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860717330 0 0) 0x7fed6006d840 con 0x7fed6006b440 2024-09-13T18:36:36.268 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5ed9d640 1 -- 172.21.15.177:0/1616755054 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6006cfb0 con 0x7fed60132d40 2024-09-13T18:36:36.269 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.264+0000 7fed5dd9b640 1 --2- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed60132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:36.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.265+0000 7fed5ed9d640 1 --2- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed60132d40 0x7fed60135130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed580097d0 tx=0x7fed5802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8911c44e369900b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:36.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.265+0000 7fed5d59a640 1 -- 172.21.15.177:0/1616755054 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5802ad90 con 0x7fed60132d40 2024-09-13T18:36:36.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.265+0000 7fed5d59a640 1 -- 172.21.15.177:0/1616755054 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed58032440 con 0x7fed60132d40 2024-09-13T18:36:36.271 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.265+0000 7fed5d59a640 1 -- 172.21.15.177:0/1616755054 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed580327d0 con 0x7fed60132d40 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.265+0000 7fed64a72640 1 -- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed60132d40 msgr2=0x7fed60135130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.265+0000 7fed64a72640 1 --2- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed60132d40 0x7fed60135130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed580097d0 tx=0x7fed5802a040 comp rx=0 tx=0).stop 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.266+0000 7fed64a72640 1 -- 172.21.15.177:0/1616755054 shutdown_connections 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.266+0000 7fed64a72640 1 --2- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed60132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.266+0000 7fed64a72640 1 --2- 172.21.15.177:0/1616755054 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed60132d40 0x7fed60135130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.266+0000 7fed64a72640 1 -- 172.21.15.177:0/1616755054 >> 172.21.15.177:0/1616755054 conn(0x7fed6009fe10 msgr2=0x7fed600a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.266+0000 7fed64a72640 1 -- 172.21.15.177:0/1616755054 shutdown_connections 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.266+0000 7fed64a72640 1 -- 172.21.15.177:0/1616755054 wait complete. 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 Processor -- start 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 -- start start 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 0x7fed6006de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.272 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed6006e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed60132d40 0x7fed6006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600702c0 con 0x7fed60132d40 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed60070aa0 con 0x7fed6006bf30 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed64a72640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed6006f4c0 con 0x7fed6006b440 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5e59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 0x7fed6006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5dd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed6006e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5ed9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed60132d40 0x7fed6006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5e59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 0x7fed6006de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36024/0 (socket says 172.21.15.177:36024) 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5ed9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed60132d40 0x7fed6006e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59764/0 (socket says 172.21.15.177:59764) 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5dd9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed6006e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54274/0 (socket says 172.21.15.177:54274) 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5e59c640 1 -- 172.21.15.177:0/3651224524 learned_addr learned my addr 172.21.15.177:0/3651224524 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:36.273 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.267+0000 7fed5e59c640 1 -- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 msgr2=0x7fed6006e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed5e59c640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed6006e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed5e59c640 1 -- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed60132d40 msgr2=0x7fed6006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed5e59c640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed60132d40 0x7fed6006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed5e59c640 1 -- 172.21.15.177:0/3651224524 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54009400 con 0x7fed6006b440 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed5e59c640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 0x7fed6006de20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c8c0 tx=0x7fed5000cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000a640 con 0x7fed6006b440 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5000a7a0 con 0x7fed6006b440 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed58009400 con 0x7fed6006b440 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000ab90 con 0x7fed6006b440 2024-09-13T18:36:36.274 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.268+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6012dd80 con 0x7fed6006b440 2024-09-13T18:36:36.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.269+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fed5000acf0 con 0x7fed6006b440 2024-09-13T18:36:36.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.269+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6012e000 con 0x7fed6006b440 2024-09-13T18:36:36.275 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.269+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fed5005c110 con 0x7fed6006b440 2024-09-13T18:36:36.276 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.273+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fed50039d90 con 0x7fed6006b440 2024-09-13T18:36:36.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.408+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed6006b820 con 0x7fed6006b440 2024-09-13T18:36:36.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.409+0000 7fed4effd640 1 -- 172.21.15.177:0/3651224524 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+9088 (secure 0 0 0) 0x7fed50018020 con 0x7fed6006b440 2024-09-13T18:36:36.412 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:36.412 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":29,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:36:35.641745+0000","last_up_change":"2024-09-13T18:36:35.641745+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":1669150124}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":1669150124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":1669150124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":1669150124}]},"public_addr":"172.21.15.177:6804/1669150124","cluster_addr":"172.21.15.177:6806/1669150124","heartbeat_back_addr":"172.21.15.177:6810/1669150124","heartbeat_front_addr":"172.21.15.177:6809/1669150124","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":2712499331}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":2712499331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":2712499331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":2712499331}]},"public_addr":"172.21.15.177:6805/2712499331","cluster_addr":"172.21.15.177:6807/2712499331","heartbeat_back_addr":"172.21.15.177:6811/2712499331","heartbeat_front_addr":"172.21.15.177:6808/2712499331","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":28,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":341082235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":341082235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":341082235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":341082235}]},"public_addr":"172.21.15.177:6800/341082235","cluster_addr":"172.21.15.177:6801/341082235","heartbeat_back_addr":"172.21.15.177:6803/341082235","heartbeat_front_addr":"172.21.15.177:6802/341082235","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":23,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":2985513301}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":2985513301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":2985513301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":2985513301}]},"public_addr":"172.21.15.177:6812/2985513301","cluster_addr":"172.21.15.177:6813/2985513301","heartbeat_back_addr":"172.21.15.177:6815/2985513301","heartbeat_front_addr":"172.21.15.177:6814/2985513301","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":22},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":23},{"osd":3,"down_stamp":"2024-09-13T18:36:29.602766+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:0/2408340411":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/2735606740":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/3625374812":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/1782828617":"2024-09-14T18:36:22.577532+0000","172.21.15.177:6816/32265":"2024-09-14T18:36:22.577532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 >> v1:172.21.15.177:6816/516212916 conn(0x7fed400475c0 legacy=0x7fed40049a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 msgr2=0x7fed6006de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 0x7fed6006de20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c8c0 tx=0x7fed5000cd80 comp rx=0 tx=0).stop 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 shutdown_connections 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed60132d40 0x7fed6006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed6006bf30 0x7fed6006e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 --2- 172.21.15.177:0/3651224524 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fed6006b440 0x7fed6006de20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 >> 172.21.15.177:0/3651224524 conn(0x7fed6009fe10 msgr2=0x7fed6009bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.410+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 shutdown_connections 2024-09-13T18:36:36.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.411+0000 7fed64a72640 1 -- 172.21.15.177:0/3651224524 wait complete. 2024-09-13T18:36:36.425 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T18:36:36.425 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T18:36:36.438 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:36:36.438 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:36:36.438 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 Processor -- start 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 -- start start 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c412a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 0x7fd8c4093140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd8c4093680 0x7fd8c4093b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c4070a60 con 0x7fd8c4093680 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4094b20 con 0x7fd8c412a000 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c9f0f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40958a0 con 0x7fd8c412a920 2024-09-13T18:36:36.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 0x7fd8c4093140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c412a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c3fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd8c4093680 0x7fd8c4093b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 0x7fd8c4093140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36038/0 (socket says 172.21.15.177:36038) 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8baffd640 1 -- 172.21.15.177:0/3281557870 learned_addr learned my addr 172.21.15.177:0/3281557870 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.529+0000 7fd8c37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c412a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54280/0 (socket says 172.21.15.177:54280) 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c37fe640 1 -- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 msgr2=0x7fd8c4093140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c37fe640 1 --2- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 0x7fd8c4093140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c37fe640 1 -- 172.21.15.177:0/3281557870 >> v2:172.21.15.177:3300/0 conn(0x7fd8c4093680 msgr2=0x7fd8c4093b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c37fe640 1 --2- 172.21.15.177:0/3281557870 >> v2:172.21.15.177:3300/0 conn(0x7fd8c4093680 0x7fd8c4093b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c37fe640 1 -- 172.21.15.177:0/3281557870 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c4094070 con 0x7fd8c412a000 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8baffd640 1 --2- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 0x7fd8c4093140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:36.534 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c3fff640 1 --2- 172.21.15.177:0/3281557870 >> v2:172.21.15.177:3300/0 conn(0x7fd8c4093680 0x7fd8c4093b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c37fe640 1 --2- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c412a3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400a940 tx=0x7fd8b402b380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b383a0303f07e4d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c2ffd640 1 -- 172.21.15.177:0/3281557870 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4033640 con 0x7fd8c412a000 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c2ffd640 1 -- 172.21.15.177:0/3281557870 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b40337a0 con 0x7fd8c412a000 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.530+0000 7fd8c2ffd640 1 -- 172.21.15.177:0/3281557870 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4033b30 con 0x7fd8c412a000 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 msgr2=0x7fd8c412a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c412a3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400a940 tx=0x7fd8b402b380 comp rx=0 tx=0).stop 2024-09-13T18:36:36.538 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/3281557870 shutdown_connections 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/3281557870 >> v2:172.21.15.177:3300/0 conn(0x7fd8c4093680 0x7fd8c4093b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c412a920 0x7fd8c4093140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/3281557870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c412a3e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/3281557870 >> 172.21.15.177:0/3281557870 conn(0x7fd8c4098b60 msgr2=0x7fd8c4098f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/3281557870 shutdown_connections 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.531+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/3281557870 wait complete. 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 Processor -- start 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 -- start start 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 0x7fd8c4120d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c4121240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c412a920 0x7fd8c4121780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4070cc0 con 0x7fd8c412a920 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4122f90 con 0x7fd8c412a000 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.532+0000 7fd8c9f0f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4123770 con 0x7fd8c4093680 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 0x7fd8c4120d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 0x7fd8c4120d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36054/0 (socket says 172.21.15.177:36054) 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c4121240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 -- 172.21.15.177:0/850792213 learned_addr learned my addr 172.21.15.177:0/850792213 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c3fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c412a920 0x7fd8c4121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 -- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 msgr2=0x7fd8c4121240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c4121240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 -- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c412a920 msgr2=0x7fd8c4121780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c412a920 0x7fd8c4121780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.540 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 -- 172.21.15.177:0/850792213 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc009a70 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c3fff640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c412a920 0x7fd8c4121780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8baffd640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c4121240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c37fe640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 0x7fd8c4120d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8b402b8a0 tx=0x7fd8b40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4031850 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b40319b0 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac0094d0 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4031d20 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.533+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b400a5a0 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.534+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fd8b403a040 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.534+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c406b4d0 con 0x7fd8c4093680 2024-09-13T18:36:36.541 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.535+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fd8b407ce50 con 0x7fd8c4093680 2024-09-13T18:36:36.542 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.538+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fd8b40475a0 con 0x7fd8c4093680 2024-09-13T18:36:36.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.727+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd8c4097ed0 con 0x7fd8c4093680 2024-09-13T18:36:36.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.727+0000 7fd8c0ff9640 1 -- 172.21.15.177:0/850792213 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+503 (secure 0 0 0) 0x7fd8b405e110 con 0x7fd8c4093680 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "mon": { 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)": 3 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": { 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)": 2 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": { 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)": 4 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.731 INFO:teuthology.orchestra.run.smithi177.stdout: "overall": { 2024-09-13T18:36:36.732 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)": 9 2024-09-13T18:36:36.732 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:36.732 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-13T18:36:36.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 >> v1:172.21.15.177:6816/516212916 conn(0x7fd8a4047590 legacy=0x7fd8a4049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 msgr2=0x7fd8c4120d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 0x7fd8c4120d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8b402b8a0 tx=0x7fd8b40046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 shutdown_connections 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c412a920 0x7fd8c4121780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8c412a000 0x7fd8c4121240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 --2- 172.21.15.177:0/850792213 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd8c4093680 0x7fd8c4120d00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 >> 172.21.15.177:0/850792213 conn(0x7fd8c4098b60 msgr2=0x7fd8c406dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 shutdown_connections 2024-09-13T18:36:36.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.729+0000 7fd8c9f0f640 1 -- 172.21.15.177:0/850792213 wait complete. 2024-09-13T18:36:36.742 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2024-09-13T18:36:36.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 Processor -- start 2024-09-13T18:36:36.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 -- start start 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a000 0x7f746412a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412f250 0x7f7464131640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7464070a60 con 0x7f746412a920 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7464094b20 con 0x7f746412f250 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f746b26b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74640958a0 con 0x7f746412a000 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f7469007640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a000 0x7f746412a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f7468806640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f746412a920 0x7f7464093140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:49354/0 (socket says 172.21.15.177:49354) 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f7468806640 1 -- 172.21.15.177:0/595172590 learned_addr learned my addr 172.21.15.177:0/595172590 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.833+0000 7f7469808640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412f250 0x7f7464131640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f7469007640 1 -- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412f250 msgr2=0x7f7464131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f7469007640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412f250 0x7f7464131640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.841 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f7469007640 1 -- 172.21.15.177:0/595172590 >> v1:172.21.15.177:6789/0 conn(0x7f746412a920 legacy=0x7f7464093140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f745bfff640 1 -- 172.21.15.177:0/595172590 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496536415 0 0) 0x7f7464070a60 con 0x7f746412a920 2024-09-13T18:36:36.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f7469007640 1 -- 172.21.15.177:0/595172590 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464097a70 con 0x7f746412a000 2024-09-13T18:36:36.842 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f7469808640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412f250 0x7f7464131640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:36.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.834+0000 7f7469007640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a000 0x7f746412a3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74600097d0 tx=0x7f746002a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f213c062153e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:36.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f745bfff640 1 -- 172.21.15.177:0/595172590 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746002ad90 con 0x7f746412a000 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f745bfff640 1 -- 172.21.15.177:0/595172590 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7460032440 con 0x7f746412a000 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f745bfff640 1 -- 172.21.15.177:0/595172590 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74600327d0 con 0x7f746412a000 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 -- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a000 msgr2=0x7f746412a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a000 0x7f746412a3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74600097d0 tx=0x7f746002a040 comp rx=0 tx=0).stop 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 -- 172.21.15.177:0/595172590 shutdown_connections 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412f250 0x7f7464131640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 --2- 172.21.15.177:0/595172590 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a000 0x7f746412a3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 -- 172.21.15.177:0/595172590 >> 172.21.15.177:0/595172590 conn(0x7f7464098b60 msgr2=0x7f7464098f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 -- 172.21.15.177:0/595172590 shutdown_connections 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.835+0000 7f746b26b640 1 -- 172.21.15.177:0/595172590 wait complete. 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 Processor -- start 2024-09-13T18:36:36.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 -- start start 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 0x7f7464127c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a920 0x7f74641281a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746412f250 0x7f74641286e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74641279a0 con 0x7f746412f250 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746412dd80 con 0x7f746412a000 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f746b26b640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74641265a0 con 0x7f746412a920 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f7469007640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 0x7f7464127c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.836+0000 7f7468806640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a920 0x7f74641281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469808640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746412f250 0x7f74641286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 0x7f7464127c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54300/0 (socket says 172.21.15.177:54300) 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 -- 172.21.15.177:0/1826597636 learned_addr learned my addr 172.21.15.177:0/1826597636 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 -- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a920 msgr2=0x7f74641281a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.846 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a920 0x7f74641281a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 -- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746412f250 msgr2=0x7f74641286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746412f250 0x7f74641286e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 -- 172.21.15.177:0/1826597636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c009a70 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7469007640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 0x7f7464127c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7460035040 tx=0x7f74600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746003caa0 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7460040070 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7460009400 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746003ce20 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.837+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7464129980 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.838+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f74600503a0 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.838+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7464131b40 con 0x7f746412a000 2024-09-13T18:36:36.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.839+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f7460042070 con 0x7f746412a000 2024-09-13T18:36:36.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.842+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f7460080270 con 0x7f746412a000 2024-09-13T18:36:36.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.981+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f746412a3e0 con 0x7f746412a000 2024-09-13T18:36:36.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.982+0000 7f7459ffb640 1 -- 172.21.15.177:0/1826597636 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v30) v1 ==== 81+0+16688 (secure 0 0 0) 0x7f746004c070 con 0x7f746412a000 2024-09-13T18:36:36.985 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:36.985 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-13T18:36:36.985 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 30, 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "af843e4a-307e-4c3a-83b8-4495d5352147", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-09-13T18:34:54.195240+0000", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-09-13T18:36:36.646729+0000", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-09-13T18:36:35.641745+0000", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-09-13T18:34:55.397217+0000", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5799936, 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-09-13T18:36:36.986 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 8, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 2, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 4, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "quincy", 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "jewel", 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "quincy", 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-09-13T18:36:36.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-09-13T18:35:04.643396+0000", 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-09-13T18:36:36.988 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-09-13T18:36:36.989 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "13", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-09-13T18:36:36.990 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-09-13T18:36:36.991 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-09-13T18:36:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 4, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 4, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.5, 2024-09-13T18:36:36.993 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 1, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 1 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-09-13T18:35:09.782965+0000", 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-09-13T18:36:36.994 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-09-13T18:36:36.995 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-09-13T18:36:36.996 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 3, 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 17, 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-09-13T18:36:36.997 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-09-13T18:36:36.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-09-13T18:36:36.999 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.5, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.5, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 1, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2.5, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2.5, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 1, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 1 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.000 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "f9b65b0a-b05a-4d13-bb43-2376cd9392da", 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 9, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 20, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 26, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 26, 2024-09-13T18:36:37.001 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 21, 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1669150124 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1669150124 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-09-13T18:36:37.003 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1669150124 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1669150124 2024-09-13T18:36:37.004 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6804/1669150124", 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6806/1669150124", 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6810/1669150124", 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6809/1669150124", 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "ab422818-9dbe-4e2f-9da7-8a78a041b09b", 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 9, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 20, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 27, 2024-09-13T18:36:37.006 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 27, 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 22, 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2712499331 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.007 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-09-13T18:36:37.008 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2712499331 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2712499331 2024-09-13T18:36:37.009 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2712499331 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6805/2712499331", 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6807/2712499331", 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6811/2712499331", 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6808/2712499331", 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.011 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "d0ca7945-14cb-40ee-b8a7-14586c366c5d", 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 9, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 22, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 28, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 29, 2024-09-13T18:36:37.012 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 23, 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 341082235 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.013 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 341082235 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-09-13T18:36:37.014 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 341082235 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.015 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 341082235 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6800/341082235", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6801/341082235", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6803/341082235", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6802/341082235", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-09-13T18:36:37.016 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "63594787-3078-431f-9ac4-e24e4bac0e7d", 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 9, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 23, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 29, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 29, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 24, 2024-09-13T18:36:37.017 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2985513301 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.018 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2985513301 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2985513301 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2985513301 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6812/2985513301", 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6813/2985513301", 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2985513301", 2024-09-13T18:36:37.020 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6814/2985513301", 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-09-13T18:36:37.021 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540138322906710015, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-09-13T18:35:00.556109+0000", 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 21 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540138322906710015, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-09-13T18:35:00.557174+0000", 2024-09-13T18:36:37.022 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 22 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540138322906710015, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-09-13T18:35:00.504393+0000", 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 23 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-09-13T18:36:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "2024-09-13T18:36:29.602766+0000", 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540138322906710015, 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-09-13T18:35:00.577686+0000", 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [], 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [], 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-09-13T18:36:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": { 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "172.21.15.177:0/2408340411": "2024-09-14T18:36:22.577532+0000", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "172.21.15.177:0/2735606740": "2024-09-14T18:36:22.577532+0000", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "172.21.15.177:0/3625374812": "2024-09-14T18:36:22.577532+0000", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "172.21.15.177:0/1782828617": "2024-09-14T18:36:22.577532+0000", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "172.21.15.177:6816/32265": "2024-09-14T18:36:22.577532+0000" 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-09-13T18:36:37.025 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [], 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [], 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-09-13T18:36:37.026 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-09-13T18:36:37.027 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-13T18:36:37.027 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-13T18:36:37.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 >> v1:172.21.15.177:6816/516212916 conn(0x7f7448047560 legacy=0x7f7448049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 msgr2=0x7f7464127c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 0x7f7464127c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7460035040 tx=0x7f74600046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 shutdown_connections 2024-09-13T18:36:37.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746412f250 0x7f74641286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f746412a920 0x7f74641281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 --2- 172.21.15.177:0/1826597636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746412a000 0x7f7464127c60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 >> 172.21.15.177:0/1826597636 conn(0x7f7464098b60 msgr2=0x7f7464124eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 shutdown_connections 2024-09-13T18:36:37.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:36.984+0000 7f746b26b640 1 -- 172.21.15.177:0/1826597636 wait complete. 2024-09-13T18:36:37.029 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-09-13T18:36:37.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.120+0000 7f36e4a61640 1 Processor -- start 2024-09-13T18:36:37.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.120+0000 7f36e4a61640 1 -- start start 2024-09-13T18:36:37.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36e4a61640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e012a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36e4a61640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f36e0092740 0x7f36e0132aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36e4a61640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e0097ce0 con 0x7f36e012ae00 2024-09-13T18:36:37.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36e4a61640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e006b910 con 0x7f36e0092740 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36e4a61640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36e00953a0 con 0x7f36e012a4e0 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e012a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36ded9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f36e0092740 0x7f36e0132aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e012a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36072/0 (socket says 172.21.15.177:36072) 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 -- 172.21.15.177:0/2783190366 learned_addr learned my addr 172.21.15.177:0/2783190366 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 -- 172.21.15.177:0/2783190366 >> v2:172.21.15.177:3300/0 conn(0x7f36e0092740 msgr2=0x7f36e0132aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 --2- 172.21.15.177:0/2783190366 >> v2:172.21.15.177:3300/0 conn(0x7f36e0092740 0x7f36e0132aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 -- 172.21.15.177:0/2783190366 >> v1:172.21.15.177:6789/0 conn(0x7f36e012ae00 legacy=0x7f36e0092030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36dd59a640 1 -- 172.21.15.177:0/2783190366 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740303730 0 0) 0x7f36e0097ce0 con 0x7f36e012ae00 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36de59c640 1 -- 172.21.15.177:0/2783190366 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0069ff0 con 0x7f36e012a4e0 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.121+0000 7f36ded9d640 1 --2- 172.21.15.177:0/2783190366 >> v2:172.21.15.177:3300/0 conn(0x7f36e0092740 0x7f36e0132aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.122+0000 7f36de59c640 1 --2- 172.21.15.177:0/2783190366 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e012a8c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009e50 tx=0x7f36d802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e7211108bc8b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.122+0000 7f36dd59a640 1 -- 172.21.15.177:0/2783190366 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d802aea0 con 0x7f36e012a4e0 2024-09-13T18:36:37.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.122+0000 7f36dd59a640 1 -- 172.21.15.177:0/2783190366 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d802d050 con 0x7f36e012a4e0 2024-09-13T18:36:37.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.122+0000 7f36dd59a640 1 -- 172.21.15.177:0/2783190366 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8031820 con 0x7f36e012a4e0 2024-09-13T18:36:37.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 -- 172.21.15.177:0/2783190366 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 msgr2=0x7f36e012a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 --2- 172.21.15.177:0/2783190366 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e012a8c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009e50 tx=0x7f36d802a680 comp rx=0 tx=0).stop 2024-09-13T18:36:37.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 -- 172.21.15.177:0/2783190366 shutdown_connections 2024-09-13T18:36:37.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 --2- 172.21.15.177:0/2783190366 >> v2:172.21.15.177:3300/0 conn(0x7f36e0092740 0x7f36e0132aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 --2- 172.21.15.177:0/2783190366 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e012a8c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 -- 172.21.15.177:0/2783190366 >> 172.21.15.177:0/2783190366 conn(0x7f36e0098b60 msgr2=0x7f36e0098f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 -- 172.21.15.177:0/2783190366 shutdown_connections 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.123+0000 7f36e4a61640 1 -- 172.21.15.177:0/2783190366 wait complete. 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 Processor -- start 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 -- start start 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36e0092740 0x7f36e0124d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e0125280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e012ae00 0x7f36e01257c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e006ed80 con 0x7f36e012ae00 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e006eaa0 con 0x7f36e0092740 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36e4a61640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36e006f060 con 0x7f36e012a4e0 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36ddd9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e0125280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36de59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36e0092740 0x7f36e0124d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e012ae00 0x7f36e01257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.124+0000 7f36ddd9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e0125280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36088/0 (socket says 172.21.15.177:36088) 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ddd9b640 1 -- 172.21.15.177:0/1546900777 learned_addr learned my addr 172.21.15.177:0/1546900777 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ded9d640 1 -- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 msgr2=0x7f36e0125280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ded9d640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e0125280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ded9d640 1 -- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36e0092740 msgr2=0x7f36e0124d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ded9d640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36e0092740 0x7f36e0124d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ded9d640 1 -- 172.21.15.177:0/1546900777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0009400 con 0x7f36e012ae00 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36de59c640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36e0092740 0x7f36e0124d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ddd9b640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e0125280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36ded9d640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e012ae00 0x7f36e01257c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e8b0 tx=0x7f36d000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004020 con 0x7f36e012ae00 2024-09-13T18:36:37.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d00041a0 con 0x7f36e012ae00 2024-09-13T18:36:37.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d8009b30 con 0x7f36e012ae00 2024-09-13T18:36:37.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d000c8e0 con 0x7f36e012ae00 2024-09-13T18:36:37.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.125+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e006f5c0 con 0x7f36e012ae00 2024-09-13T18:36:37.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.126+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f36d000ca40 con 0x7f36e012ae00 2024-09-13T18:36:37.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.126+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e012a8c0 con 0x7f36e012ae00 2024-09-13T18:36:37.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.126+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f36d005c7c0 con 0x7f36e012ae00 2024-09-13T18:36:37.133 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.130+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f36d003a3a0 con 0x7f36e012ae00 2024-09-13T18:36:37.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.285+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f36e0067910 con 0x7f36e012ae00 2024-09-13T18:36:37.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.285+0000 7f36d6ffd640 1 -- 172.21.15.177:0/1546900777 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7f36d0017070 con 0x7f36e012ae00 2024-09-13T18:36:37.289 INFO:teuthology.orchestra.run.smithi177.stdout:e1 2024-09-13T18:36:37.289 INFO:teuthology.orchestra.run.smithi177.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T18:36:37.289 INFO:teuthology.orchestra.run.smithi177.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:36:37.289 INFO:teuthology.orchestra.run.smithi177.stdout:legacy client fscid: -1 2024-09-13T18:36:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:37.289 INFO:teuthology.orchestra.run.smithi177.stdout:No filesystems configured 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:dumped fsmap epoch 1 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 >> v1:172.21.15.177:6816/516212916 conn(0x7f36c0047560 legacy=0x7f36c0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e012ae00 msgr2=0x7f36e01257c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e012ae00 0x7f36e01257c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e8b0 tx=0x7f36d000ed70 comp rx=0 tx=0).stop 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 shutdown_connections 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e012ae00 0x7f36e01257c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f36e012a4e0 0x7f36e0125280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 --2- 172.21.15.177:0/1546900777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36e0092740 0x7f36e0124d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 >> 172.21.15.177:0/1546900777 conn(0x7f36e0098b60 msgr2=0x7f36e009a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 shutdown_connections 2024-09-13T18:36:37.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.287+0000 7f36e4a61640 1 -- 172.21.15.177:0/1546900777 wait complete. 2024-09-13T18:36:37.300 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release reef' 2024-09-13T18:36:37.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.392+0000 7fd2073d4640 1 Processor -- start 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd2073d4640 1 -- start start 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd2073d4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 0x7fd20012a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd2073d4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd20012f250 0x7fd200131640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd2073d4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd200070a60 con 0x7fd20012a920 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd2073d4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd200094b20 con 0x7fd20012f250 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd2073d4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000958a0 con 0x7fd20012a000 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 0x7fd20012a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 0x7fd20012a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36092/0 (socket says 172.21.15.177:36092) 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205971640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd20012f250 0x7fd200131640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 -- 172.21.15.177:0/2074151671 learned_addr learned my addr 172.21.15.177:0/2074151671 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 -- 172.21.15.177:0/2074151671 >> v2:172.21.15.177:3300/0 conn(0x7fd20012f250 msgr2=0x7fd200131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:37.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 --2- 172.21.15.177:0/2074151671 >> v2:172.21.15.177:3300/0 conn(0x7fd20012f250 0x7fd200131640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 -- 172.21.15.177:0/2074151671 >> v1:172.21.15.177:6789/0 conn(0x7fd20012a920 legacy=0x7fd200093140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205170640 1 -- 172.21.15.177:0/2074151671 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd200097a70 con 0x7fd20012a000 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd1fffff640 1 -- 172.21.15.177:0/2074151671 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394654403 0 0) 0x7fd200070a60 con 0x7fd20012a920 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.393+0000 7fd205971640 1 --2- 172.21.15.177:0/2074151671 >> v2:172.21.15.177:3300/0 conn(0x7fd20012f250 0x7fd200131640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.394+0000 7fd205170640 1 --2- 172.21.15.177:0/2074151671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 0x7fd20012a3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098a0 tx=0x7fd1f802a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9452c0a238813a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.394+0000 7fd1fffff640 1 -- 172.21.15.177:0/2074151671 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f802ae70 con 0x7fd20012a000 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.394+0000 7fd1fffff640 1 -- 172.21.15.177:0/2074151671 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f8032440 con 0x7fd20012a000 2024-09-13T18:36:37.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.394+0000 7fd1fffff640 1 -- 172.21.15.177:0/2074151671 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80327d0 con 0x7fd20012a000 2024-09-13T18:36:37.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 -- 172.21.15.177:0/2074151671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 msgr2=0x7fd20012a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 --2- 172.21.15.177:0/2074151671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 0x7fd20012a3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098a0 tx=0x7fd1f802a160 comp rx=0 tx=0).stop 2024-09-13T18:36:37.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 -- 172.21.15.177:0/2074151671 shutdown_connections 2024-09-13T18:36:37.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 --2- 172.21.15.177:0/2074151671 >> v2:172.21.15.177:3300/0 conn(0x7fd20012f250 0x7fd200131640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 --2- 172.21.15.177:0/2074151671 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012a000 0x7fd20012a3e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 -- 172.21.15.177:0/2074151671 >> 172.21.15.177:0/2074151671 conn(0x7fd200098b60 msgr2=0x7fd200098f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 -- 172.21.15.177:0/2074151671 shutdown_connections 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.395+0000 7fd2073d4640 1 -- 172.21.15.177:0/2074151671 wait complete. 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 Processor -- start 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 -- start start 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 0x7fd20006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd20012a920 0x7fd20006f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012f250 0x7fd2000913e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd200070c10 con 0x7fd20012a000 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20008f460 con 0x7fd20012a920 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd2073d4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd200091120 con 0x7fd20012f250 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd205170640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 0x7fd20006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd205170640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 0x7fd20006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59850/0 (socket says 172.21.15.177:59850) 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd205170640 1 -- 172.21.15.177:0/817129536 learned_addr learned my addr 172.21.15.177:0/817129536 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd205971640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012f250 0x7fd2000913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.396+0000 7fd20496f640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd20012a920 0x7fd20006f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd205170640 1 -- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012f250 msgr2=0x7fd2000913e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd205170640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012f250 0x7fd2000913e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd205170640 1 -- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd20012a920 msgr2=0x7fd20006f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd205170640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd20012a920 0x7fd20006f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd205170640 1 -- 172.21.15.177:0/817129536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0009a70 con 0x7fd20012a000 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd20496f640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd20012a920 0x7fd20006f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd205170640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 0x7fd20006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80090b0 tx=0x7fd1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f803d830 con 0x7fd20012a000 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f803d990 con 0x7fd20012a000 2024-09-13T18:36:37.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f80094d0 con 0x7fd20012a000 2024-09-13T18:36:37.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f803dd00 con 0x7fd20012a000 2024-09-13T18:36:37.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.397+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd200090770 con 0x7fd20012a000 2024-09-13T18:36:37.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.398+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fd1f8039040 con 0x7fd20012a000 2024-09-13T18:36:37.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.398+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20012a3e0 con 0x7fd20012a000 2024-09-13T18:36:37.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.398+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fd1f807d380 con 0x7fd20012a000 2024-09-13T18:36:37.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.402+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fd1f8037070 con 0x7fd20012a000 2024-09-13T18:36:37.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.544+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd require-osd-release", "release": "reef"} v 0) v1 -- 0x7fd2000904b0 con 0x7fd20012a000 2024-09-13T18:36:37.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.656+0000 7fd1fdffb640 1 -- 172.21.15.177:0/817129536 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "reef"}]=0 v31) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd1f803de60 con 0x7fd20012a000 2024-09-13T18:36:37.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 >> v1:172.21.15.177:6816/516212916 conn(0x7fd1ec047560 legacy=0x7fd1ec049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 msgr2=0x7fd20006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 0x7fd20006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80090b0 tx=0x7fd1f80046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 shutdown_connections 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd20012f250 0x7fd2000913e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd20012a920 0x7fd20006f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 --2- 172.21.15.177:0/817129536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20012a000 0x7fd20006edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 >> 172.21.15.177:0/817129536 conn(0x7fd200098b60 msgr2=0x7fd20012e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 shutdown_connections 2024-09-13T18:36:37.662 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.658+0000 7fd2073d4640 1 -- 172.21.15.177:0/817129536 wait complete. 2024-09-13T18:36:37.672 DEBUG:teuthology.orchestra.run.smithi177:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2024-09-13T18:36:37.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.764+0000 7f399ba3a640 1 Processor -- start 2024-09-13T18:36:37.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.764+0000 7f399ba3a640 1 -- start start 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f399ba3a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 0x7f399412dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f399ba3a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 0x7f399409b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f399ba3a640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f399409bf30 0x7f399409c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f399ba3a640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3994097550 con 0x7f399409bf30 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f399ba3a640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399412d720 con 0x7f399412d870 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f399ba3a640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3994073810 con 0x7f399412e190 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f39997d6640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 0x7f399412dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3998fd5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 0x7f399409b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.770 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f399409bf30 0x7f399409c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f399409bf30 0x7f399409c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59856/0 (socket says 172.21.15.177:59856) 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f39997d6640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 0x7f399412dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:54324/0 (socket says 172.21.15.177:54324) 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3998fd5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 0x7f399409b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36112/0 (socket says 172.21.15.177:36112) 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 -- 172.21.15.177:0/1151411725 learned_addr learned my addr 172.21.15.177:0/1151411725 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 -- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 msgr2=0x7f399409b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 0x7f399409b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 -- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 msgr2=0x7f399412dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 0x7f399412dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3999fd7640 1 -- 172.21.15.177:0/1151411725 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399412e570 con 0x7f399409bf30 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f39997d6640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 0x7f399412dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.765+0000 7f3998fd5640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 0x7f399409b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.766+0000 7f3999fd7640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399409bf30 0x7f399409c3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f398c009e50 tx=0x7f398c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d63d7be16443ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:37.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.766+0000 7f398bfff640 1 -- 172.21.15.177:0/1151411725 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c031600 con 0x7f399409bf30 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.766+0000 7f398bfff640 1 -- 172.21.15.177:0/1151411725 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c02d050 con 0x7f399409bf30 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.766+0000 7f398bfff640 1 -- 172.21.15.177:0/1151411725 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c031980 con 0x7f399409bf30 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 -- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399409bf30 msgr2=0x7f399409c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399409bf30 0x7f399409c3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f398c009e50 tx=0x7f398c02a560 comp rx=0 tx=0).stop 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 -- 172.21.15.177:0/1151411725 shutdown_connections 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399409bf30 0x7f399409c3e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399412e190 0x7f399409b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 --2- 172.21.15.177:0/1151411725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412d870 0x7f399412dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 -- 172.21.15.177:0/1151411725 >> 172.21.15.177:0/1151411725 conn(0x7f399406b9b0 msgr2=0x7f399406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 -- 172.21.15.177:0/1151411725 shutdown_connections 2024-09-13T18:36:37.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.767+0000 7f399ba3a640 1 -- 172.21.15.177:0/1151411725 wait complete. 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 Processor -- start 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 -- start start 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 0x7f399412f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399412d870 0x7f399412fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412e190 0x7f3994130120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994132500 con 0x7f399412d870 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39941318e0 con 0x7f399412e190 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f399ba3a640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3994131340 con 0x7f399409bf30 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f39997d6640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 0x7f399412f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3999fd7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412e190 0x7f3994130120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3998fd5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399412d870 0x7f399412fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f39997d6640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 0x7f399412f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36124/0 (socket says 172.21.15.177:36124) 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f39997d6640 1 -- 172.21.15.177:0/2011233006 learned_addr learned my addr 172.21.15.177:0/2011233006 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3999fd7640 1 -- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 msgr2=0x7f399412f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3999fd7640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 0x7f399412f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3999fd7640 1 -- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399412d870 msgr2=0x7f399412fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3999fd7640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399412d870 0x7f399412fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3999fd7640 1 -- 172.21.15.177:0/2011233006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009400 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f39997d6640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 0x7f399412f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.768+0000 7f3998fd5640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399412d870 0x7f399412fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.769+0000 7f3999fd7640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412e190 0x7f3994130120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3994097b10 tx=0x7f398c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.769+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c03b810 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.769+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c03b970 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.769+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3984009400 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.769+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c03bd20 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.769+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c009b30 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.770+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f398c03a3a0 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.770+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3994128140 con 0x7f399412e190 2024-09-13T18:36:37.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.770+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f398c07d460 con 0x7f399412e190 2024-09-13T18:36:37.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.774+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f3994128140 con 0x7f399412e190 2024-09-13T18:36:37.910 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.907+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f3994128140 con 0x7f399412e190 2024-09-13T18:36:37.911 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.908+0000 7f3989ffb640 1 -- 172.21.15.177:0/2011233006 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v31) v1 ==== 59+0+9 (secure 0 0 0) 0x7f3994128140 con 0x7f399412e190 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 >> v1:172.21.15.177:6816/516212916 conn(0x7f3978047560 legacy=0x7f3978049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412e190 msgr2=0x7f3994130120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412e190 0x7f3994130120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3994097b10 tx=0x7f398c004a40 comp rx=0 tx=0).stop 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 shutdown_connections 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f399412e190 0x7f3994130120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f399412d870 0x7f399412fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 --2- 172.21.15.177:0/2011233006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f399409bf30 0x7f399412f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 >> 172.21.15.177:0/2011233006 conn(0x7f399406b9b0 msgr2=0x7f3994099700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:37.914 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 shutdown_connections 2024-09-13T18:36:37.914 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.910+0000 7f399ba3a640 1 -- 172.21.15.177:0/2011233006 wait complete. 2024-09-13T18:36:37.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 Processor -- start 2024-09-13T18:36:37.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 -- start start 2024-09-13T18:36:37.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffbac094250 0x7ffbac094630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 0x7ffbac094b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:37.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac06bb10 con 0x7ffbac092030 2024-09-13T18:36:37.997 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac095be0 con 0x7ffbac094250 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.993+0000 7ffbb0af0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac06b550 con 0x7ffbac0955c0 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 0x7ffbac094b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffbaa59c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffbac094250 0x7ffbac094630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 0x7ffbac094b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:36134/0 (socket says 172.21.15.177:36134) 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffbaa59c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffbac094250 0x7ffbac094630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59878/0 (socket says 172.21.15.177:59878) 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 -- 172.21.15.177:0/2099677658 learned_addr learned my addr 172.21.15.177:0/2099677658 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffbaad9d640 1 --1- 172.21.15.177:0/2099677658 >> v1:172.21.15.177:6789/0 conn(0x7ffbac092030 0x7ffbac092480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:49378/0 (socket says 172.21.15.177:49378) 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 -- 172.21.15.177:0/2099677658 >> v2:172.21.15.177:3300/0 conn(0x7ffbac094250 msgr2=0x7ffbac094630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 --2- 172.21.15.177:0/2099677658 >> v2:172.21.15.177:3300/0 conn(0x7ffbac094250 0x7ffbac094630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 -- 172.21.15.177:0/2099677658 >> v1:172.21.15.177:6789/0 conn(0x7ffbac092030 legacy=0x7ffbac092480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:37.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.994+0000 7ffba9d9b640 1 -- 172.21.15.177:0/2099677658 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0950b0 con 0x7ffbac0955c0 2024-09-13T18:36:38.001 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffba9d9b640 1 --2- 172.21.15.177:0/2099677658 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 0x7ffbac094b70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009870 tx=0x7ffba002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf904b48363eae25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:38.001 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffba959a640 1 -- 172.21.15.177:0/2099677658 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba002ae80 con 0x7ffbac0955c0 2024-09-13T18:36:38.001 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffba959a640 1 -- 172.21.15.177:0/2099677658 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba0032440 con 0x7ffbac0955c0 2024-09-13T18:36:38.001 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffba959a640 1 -- 172.21.15.177:0/2099677658 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba00327b0 con 0x7ffbac0955c0 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2099677658 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 msgr2=0x7ffbac094b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2099677658 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 0x7ffbac094b70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009870 tx=0x7ffba002a130 comp rx=0 tx=0).stop 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2099677658 shutdown_connections 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2099677658 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac0955c0 0x7ffbac094b70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2099677658 >> v2:172.21.15.177:3300/0 conn(0x7ffbac094250 0x7ffbac094630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2099677658 >> 172.21.15.177:0/2099677658 conn(0x7ffbac098e20 msgr2=0x7ffbac0965d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2099677658 shutdown_connections 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.995+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2099677658 wait complete. 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.996+0000 7ffbb0af0640 1 Processor -- start 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.996+0000 7ffbb0af0640 1 -- start start 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbac092030 0x7ffbac1c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac094250 0x7ffbac1c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 0x7ffbac090b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1227c0 con 0x7ffbac0955c0 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0716a0 con 0x7ffbac092030 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac121b50 con 0x7ffbac094250 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 0x7ffbac090b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffba9d9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac094250 0x7ffbac1c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaa59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbac092030 0x7ffbac1c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 0x7ffbac090b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:59888/0 (socket says 172.21.15.177:59888) 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 -- 172.21.15.177:0/2173225148 learned_addr learned my addr 172.21.15.177:0/2173225148 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 -- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac094250 msgr2=0x7ffbac1c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac094250 0x7ffbac1c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 -- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbac092030 msgr2=0x7ffbac1c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbac092030 0x7ffbac1c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.003 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 -- 172.21.15.177:0/2173225148 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00094d0 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffba9d9b640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac094250 0x7ffbac1c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbaad9d640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 0x7ffbac090b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c780 tx=0x7ffba400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400ce90 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba400a5e0 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac122f90 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400a990 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.997+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac1c3290 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.998+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7ffba400aaf0 con 0x7ffbac0955c0 2024-09-13T18:36:38.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.998+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac1224a0 con 0x7ffbac0955c0 2024-09-13T18:36:38.005 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:37.999+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3027+0+0 (secure 0 0 0) 0x7ffba405c070 con 0x7ffbac0955c0 2024-09-13T18:36:38.005 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.002+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7ffba4039ce0 con 0x7ffbac0955c0 2024-09-13T18:36:38.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.136+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7ffbac097f60 con 0x7ffbac0955c0 2024-09-13T18:36:38.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.663+0000 7ffb9affd640 1 -- 172.21.15.177:0/2173225148 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v32) v1 ==== 153+0+0 (secure 0 0 0) 0x7ffba4017020 con 0x7ffbac0955c0 2024-09-13T18:36:38.667 INFO:teuthology.orchestra.run.smithi177.stderr:set pool 1 pg_autoscale_mode to off 2024-09-13T18:36:38.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.665+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 >> v1:172.21.15.177:6816/516212916 conn(0x7ffb8c047560 legacy=0x7ffb8c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.665+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 msgr2=0x7ffbac090b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.665+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 0x7ffbac090b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c780 tx=0x7ffba400cc40 comp rx=0 tx=0).stop 2024-09-13T18:36:38.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 shutdown_connections 2024-09-13T18:36:38.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbac0955c0 0x7ffbac090b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7ffbac094250 0x7ffbac1c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 --2- 172.21.15.177:0/2173225148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbac092030 0x7ffbac1c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 >> 172.21.15.177:0/2173225148 conn(0x7ffbac098e20 msgr2=0x7ffbac09a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:38.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 shutdown_connections 2024-09-13T18:36:38.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.666+0000 7ffbb0af0640 1 -- 172.21.15.177:0/2173225148 wait complete. 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 Processor -- start 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 -- start start 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 0x7f0cb406bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb409a220 con 0x7f0cb406b7b0 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb406b3f0 con 0x7f0cb412dba0 2024-09-13T18:36:38.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb9718640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb409c880 con 0x7f0cb406c0d0 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb2ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 0x7f0cb406bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb27fc640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb2ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 0x7f0cb406bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45702/0 (socket says 172.21.15.177:45702) 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.751+0000 7f0cb2ffd640 1 -- 172.21.15.177:0/740297542 learned_addr learned my addr 172.21.15.177:0/740297542 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb37fe640 1 -- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 msgr2=0x7f0cb412d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb37fe640 1 --2- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb37fe640 1 -- 172.21.15.177:0/740297542 >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 msgr2=0x7f0cb406bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb37fe640 1 --2- 172.21.15.177:0/740297542 >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 0x7f0cb406bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb37fe640 1 -- 172.21.15.177:0/740297542 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4097550 con 0x7f0cb412dba0 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb2ffd640 1 --2- 172.21.15.177:0/740297542 >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 0x7f0cb406bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:38.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb27fc640 1 --2- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:38.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.752+0000 7f0cb37fe640 1 --2- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412e050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009dc0 tx=0x7f0cac02a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d524753b4490a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:38.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb1ffb640 1 -- 172.21.15.177:0/740297542 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac007740 con 0x7f0cb412dba0 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb1ffb640 1 -- 172.21.15.177:0/740297542 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac02d050 con 0x7f0cb412dba0 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb1ffb640 1 -- 172.21.15.177:0/740297542 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac007ac0 con 0x7f0cb412dba0 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 -- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 msgr2=0x7f0cb412e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 --2- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412e050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009dc0 tx=0x7f0cac02a4d0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 -- 172.21.15.177:0/740297542 shutdown_connections 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 --2- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412e050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 --2- 172.21.15.177:0/740297542 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 --2- 172.21.15.177:0/740297542 >> v2:172.21.15.177:3300/0 conn(0x7f0cb406b7b0 0x7f0cb406bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 -- 172.21.15.177:0/740297542 >> 172.21.15.177:0/740297542 conn(0x7f0cb409d180 msgr2=0x7f0cb409aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 -- 172.21.15.177:0/740297542 shutdown_connections 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.753+0000 7f0cb9718640 1 -- 172.21.15.177:0/740297542 wait complete. 2024-09-13T18:36:38.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.754+0000 7f0cb9718640 1 Processor -- start 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.754+0000 7f0cb9718640 1 -- start start 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cb406b7b0 0x7f0cb412f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4132010 con 0x7f0cb406b7b0 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4130be0 con 0x7f0cb412dba0 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4131d30 con 0x7f0cb406c0d0 2024-09-13T18:36:38.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb27fc640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56752/0 (socket says 172.21.15.177:56752) 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb27fc640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60318/0 (socket says 172.21.15.177:60318) 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 -- 172.21.15.177:0/3551569006 learned_addr learned my addr 172.21.15.177:0/3551569006 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb2ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cb406b7b0 0x7f0cb412f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 -- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 msgr2=0x7f0cb412f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 -- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cb406b7b0 msgr2=0x7f0cb412f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cb406b7b0 0x7f0cb412f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 -- 172.21.15.177:0/3551569006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40094d0 con 0x7f0cb412dba0 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb2ffd640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cb406b7b0 0x7f0cb412f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb27fc640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb37fe640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412fb90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02a9f0 tx=0x7f0cac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:38.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac03c830 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac03c990 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac03cd20 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca8009400 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.755+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac009a70 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.756+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0cac031400 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.756+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb409a3b0 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.757+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f0cac07e2e0 con 0x7f0cb412dba0 2024-09-13T18:36:38.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.760+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0cac07d390 con 0x7f0cb412dba0 2024-09-13T18:36:38.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:38.894+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f0cb412d660 con 0x7f0cb412dba0 2024-09-13T18:36:39.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.665+0000 7f0c9b7fe640 1 -- 172.21.15.177:0/3551569006 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v33) v1 ==== 152+0+0 (secure 0 0 0) 0x7f0cac07d390 con 0x7f0cb412dba0 2024-09-13T18:36:39.669 INFO:teuthology.orchestra.run.smithi177.stderr:set pool 2 pg_autoscale_mode to off 2024-09-13T18:36:39.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 >> v1:172.21.15.177:6816/516212916 conn(0x7f0ca0047560 legacy=0x7f0ca0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 msgr2=0x7f0cb412fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412fb90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02a9f0 tx=0x7f0cac004a40 comp rx=0 tx=0).stop 2024-09-13T18:36:39.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 shutdown_connections 2024-09-13T18:36:39.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cb412dba0 0x7f0cb412fb90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0cb406c0d0 0x7f0cb412f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 --2- 172.21.15.177:0/3551569006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cb406b7b0 0x7f0cb412f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 >> 172.21.15.177:0/3551569006 conn(0x7f0cb409d180 msgr2=0x7f0cb4098df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:39.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 shutdown_connections 2024-09-13T18:36:39.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.668+0000 7f0cb9718640 1 -- 172.21.15.177:0/3551569006 wait complete. 2024-09-13T18:36:39.686 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-13T18:36:39.696 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T18:36:39.696 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T18:36:39.696 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T18:36:39.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 Processor -- start 2024-09-13T18:36:39.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 -- start start 2024-09-13T18:36:39.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5820095270 0x7f5820095650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f5820095b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 0x7f582006b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582006db30 con 0x7f582006b1d0 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582006e080 con 0x7f5820095270 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.796+0000 7f5825bae640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582006e390 con 0x7f582006cca0 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f5820095b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 0x7f582006b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5820095270 0x7f5820095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 0x7f582006b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45722/0 (socket says 172.21.15.177:45722) 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f5820095b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60320/0 (socket says 172.21.15.177:60320) 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581ffff640 1 -- 172.21.15.177:0/1914259624 learned_addr learned my addr 172.21.15.177:0/1914259624 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:39.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 -- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5820095270 msgr2=0x7f5820095650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:39.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 --2- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5820095270 0x7f5820095650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 -- 172.21.15.177:0/1914259624 >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 msgr2=0x7f582006b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:39.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 --2- 172.21.15.177:0/1914259624 >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 0x7f582006b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581effd640 1 -- 172.21.15.177:0/1914259624 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582006bbc0 con 0x7f582006cca0 2024-09-13T18:36:39.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581f7fe640 1 --2- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5820095270 0x7f5820095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:39.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.797+0000 7f581ffff640 1 --2- 172.21.15.177:0/1914259624 >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 0x7f582006b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:39.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f581effd640 1 --2- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f5820095b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5814009d90 tx=0x7f581402a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b06a5a916b1aad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:39.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f581e7fc640 1 -- 172.21.15.177:0/1914259624 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814007740 con 0x7f582006cca0 2024-09-13T18:36:39.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f581e7fc640 1 -- 172.21.15.177:0/1914259624 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581402d050 con 0x7f582006cca0 2024-09-13T18:36:39.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f581e7fc640 1 -- 172.21.15.177:0/1914259624 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814007ac0 con 0x7f582006cca0 2024-09-13T18:36:39.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 -- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 msgr2=0x7f5820095b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 --2- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f5820095b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5814009d90 tx=0x7f581402a490 comp rx=0 tx=0).stop 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 -- 172.21.15.177:0/1914259624 shutdown_connections 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 --2- 172.21.15.177:0/1914259624 >> v2:172.21.15.177:3300/0 conn(0x7f582006b1d0 0x7f582006b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 --2- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f5820095b90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 --2- 172.21.15.177:0/1914259624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5820095270 0x7f5820095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 -- 172.21.15.177:0/1914259624 >> 172.21.15.177:0/1914259624 conn(0x7f5820098ba0 msgr2=0x7f5820096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 -- 172.21.15.177:0/1914259624 shutdown_connections 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.798+0000 7f5825bae640 1 -- 172.21.15.177:0/1914259624 wait complete. 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.799+0000 7f5825bae640 1 Processor -- start 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.799+0000 7f5825bae640 1 -- start start 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f5825bae640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 0x7f5820092fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f5825bae640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f58200934e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:39.806 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f5825bae640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5820095270 0x7f5820093a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f5825bae640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820094ae0 con 0x7f5820095270 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f5825bae640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5820121fd0 con 0x7f582006b1d0 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f5825bae640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58201222e0 con 0x7f582006cca0 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 0x7f5820092fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f58200934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 0x7f5820092fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56770/0 (socket says 172.21.15.177:56770) 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 -- 172.21.15.177:0/3105803641 learned_addr learned my addr 172.21.15.177:0/3105803641 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581ffff640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5820095270 0x7f5820093a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 -- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 msgr2=0x7f58200934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f58200934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 -- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5820095270 msgr2=0x7f5820093a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5820095270 0x7f5820093a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 -- 172.21.15.177:0/3105803641 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581800d970 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581ffff640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5820095270 0x7f5820093a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.800+0000 7f581f7fe640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 0x7f5820092fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f581000fde0 tx=0x7f581000d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.801+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810018640 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.801+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58100187a0 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.801+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581000b4b0 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.801+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810018b30 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.801+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814009a70 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.802+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5810018c90 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.802+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582006b680 con 0x7f582006b1d0 2024-09-13T18:36:39.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.802+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f581005d330 con 0x7f582006b1d0 2024-09-13T18:36:39.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.805+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f581003afa0 con 0x7f582006b1d0 2024-09-13T18:36:39.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.969+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5820121620 con 0x7f582006b1d0 2024-09-13T18:36:39.974 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.972+0000 7f5807fff640 1 -- 172.21.15.177:0/3105803641 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v10) v1 ==== 74+0+107695 (secure 0 0 0) 0x7f581001d020 con 0x7f582006b1d0 2024-09-13T18:36:39.975 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:39.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 >> v1:172.21.15.177:6816/516212916 conn(0x7f580c047560 legacy=0x7f580c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 msgr2=0x7f5820092fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:39.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 0x7f5820092fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f581000fde0 tx=0x7f581000d720 comp rx=0 tx=0).stop 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 shutdown_connections 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5820095270 0x7f5820093a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f582006cca0 0x7f58200934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 --2- 172.21.15.177:0/3105803641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582006b1d0 0x7f5820092fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 >> 172.21.15.177:0/3105803641 conn(0x7f5820098ba0 msgr2=0x7f582006f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 shutdown_connections 2024-09-13T18:36:39.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:39.973+0000 7f5825bae640 1 -- 172.21.15.177:0/3105803641 wait complete. 2024-09-13T18:36:39.986 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":10,"flags":0,"active_gid":14106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6816","nonce":516212916}]},"active_addr":"172.21.15.177:6816/516212916","active_change":"2024-09-13T18:36:22.577714+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14103,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":20,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3236139501}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":1516939243}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":747125984}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2652990054}]}]} 2024-09-13T18:36:39.989 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T18:36:39.989 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T18:36:39.990 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 Processor -- start 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 -- start start 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ab806bf30 0x7f3ab8132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 0x7f3ab8135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab806d840 con 0x7f3ab806b440 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab806c4a0 con 0x7f3ab806bf30 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3abe118640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab806db60 con 0x7f3ab8132d40 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 0x7f3ab8135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3aaffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ab806bf30 0x7f3ab8132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 0x7f3ab8135130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56772/0 (socket says 172.21.15.177:56772) 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.084+0000 7f3ab7fff640 1 -- 172.21.15.177:0/3786693691 learned_addr learned my addr 172.21.15.177:0/3786693691 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3aaffff640 1 -- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 msgr2=0x7f3ab8135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3aaffff640 1 --2- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 0x7f3ab8135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3aaffff640 1 -- 172.21.15.177:0/3786693691 >> v1:172.21.15.177:6789/0 conn(0x7f3ab806b440 legacy=0x7f3ab806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3aaffff640 1 -- 172.21.15.177:0/3786693691 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab806cfb0 con 0x7f3ab806bf30 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3ab6ffd640 1 -- 172.21.15.177:0/3786693691 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936085224 0 0) 0x7f3ab806d840 con 0x7f3ab806b440 2024-09-13T18:36:40.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3ab7fff640 1 --2- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 0x7f3ab8135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3aaffff640 1 --2- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab8132800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00097a0 tx=0x7f3aa002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1487a6735d4063 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.085+0000 7f3ab6ffd640 1 -- 172.21.15.177:0/3786693691 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa002adb0 con 0x7f3ab806bf30 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3ab6ffd640 1 -- 172.21.15.177:0/3786693691 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0032440 con 0x7f3ab806bf30 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3ab6ffd640 1 -- 172.21.15.177:0/3786693691 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00327d0 con 0x7f3ab806bf30 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 -- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 msgr2=0x7f3ab8132800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 --2- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab8132800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00097a0 tx=0x7f3aa002a060 comp rx=0 tx=0).stop 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 -- 172.21.15.177:0/3786693691 shutdown_connections 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 --2- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab8132d40 0x7f3ab8135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.092 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 --2- 172.21.15.177:0/3786693691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab8132800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 -- 172.21.15.177:0/3786693691 >> 172.21.15.177:0/3786693691 conn(0x7f3ab809fe10 msgr2=0x7f3ab80a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 -- 172.21.15.177:0/3786693691 shutdown_connections 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.086+0000 7f3abe118640 1 -- 172.21.15.177:0/3786693691 wait complete. 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 Processor -- start 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 -- start start 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 0x7f3ab806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3ab8132d40 0x7f3ab806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8132540 con 0x7f3ab806bf30 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab806f1b0 con 0x7f3ab806b440 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3abe118640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab812df60 con 0x7f3ab8132d40 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 0x7f3ab806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.093 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 0x7f3ab806de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56786/0 (socket says 172.21.15.177:56786) 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3ab77fe640 1 -- 172.21.15.177:0/852791564 learned_addr learned my addr 172.21.15.177:0/852791564 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.087+0000 7f3ab7fff640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3ab8132d40 0x7f3ab806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3aaffff640 1 -- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3ab8132d40 msgr2=0x7f3ab806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3aaffff640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3ab8132d40 0x7f3ab806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3aaffff640 1 -- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 msgr2=0x7f3ab806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3aaffff640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 0x7f3ab806de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3aaffff640 1 -- 172.21.15.177:0/852791564 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0009b30 con 0x7f3ab806bf30 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3ab77fe640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 0x7f3ab806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3aaffff640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab806e360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0031000 tx=0x7f3aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa003d850 con 0x7f3ab806bf30 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa003d9b0 con 0x7f3ab806bf30 2024-09-13T18:36:40.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa0009400 con 0x7f3ab806bf30 2024-09-13T18:36:40.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa003dd60 con 0x7f3ab806bf30 2024-09-13T18:36:40.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.088+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab8070ba0 con 0x7f3ab806bf30 2024-09-13T18:36:40.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.089+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab806b820 con 0x7f3ab806bf30 2024-09-13T18:36:40.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.089+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3aa0039040 con 0x7f3ab806bf30 2024-09-13T18:36:40.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.090+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f3aa007d120 con 0x7f3ab806bf30 2024-09-13T18:36:40.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.093+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f3aa0047d00 con 0x7f3ab806bf30 2024-09-13T18:36:40.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.230+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ab81265e0 con 0x7f3ab806bf30 2024-09-13T18:36:40.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.230+0000 7f3ab4ff9640 1 -- 172.21.15.177:0/852791564 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+9086 (secure 0 0 0) 0x7f3aa0041090 con 0x7f3ab806bf30 2024-09-13T18:36:40.234 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:40.234 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":33,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:36:39.660415+0000","last_up_change":"2024-09-13T18:36:35.641745+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":1669150124}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":1669150124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":1669150124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":1669150124}]},"public_addr":"172.21.15.177:6804/1669150124","cluster_addr":"172.21.15.177:6806/1669150124","heartbeat_back_addr":"172.21.15.177:6810/1669150124","heartbeat_front_addr":"172.21.15.177:6809/1669150124","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":2712499331}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":2712499331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":2712499331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":2712499331}]},"public_addr":"172.21.15.177:6805/2712499331","cluster_addr":"172.21.15.177:6807/2712499331","heartbeat_back_addr":"172.21.15.177:6811/2712499331","heartbeat_front_addr":"172.21.15.177:6808/2712499331","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":29,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":341082235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":341082235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":341082235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":341082235}]},"public_addr":"172.21.15.177:6800/341082235","cluster_addr":"172.21.15.177:6801/341082235","heartbeat_back_addr":"172.21.15.177:6803/341082235","heartbeat_front_addr":"172.21.15.177:6802/341082235","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":29,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":2985513301}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":2985513301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":2985513301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":2985513301}]},"public_addr":"172.21.15.177:6812/2985513301","cluster_addr":"172.21.15.177:6813/2985513301","heartbeat_back_addr":"172.21.15.177:6815/2985513301","heartbeat_front_addr":"172.21.15.177:6814/2985513301","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":22},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":23},{"osd":3,"down_stamp":"2024-09-13T18:36:29.602766+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:0/2408340411":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/2735606740":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/3625374812":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/1782828617":"2024-09-14T18:36:22.577532+0000","172.21.15.177:6816/32265":"2024-09-14T18:36:22.577532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:40.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 >> v1:172.21.15.177:6816/516212916 conn(0x7f3a98047560 legacy=0x7f3a98049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 msgr2=0x7f3ab806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab806e360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0031000 tx=0x7f3aa00046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 shutdown_connections 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3ab8132d40 0x7f3ab806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab806bf30 0x7f3ab806e360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 --2- 172.21.15.177:0/852791564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab806b440 0x7f3ab806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 >> 172.21.15.177:0/852791564 conn(0x7f3ab809fe10 msgr2=0x7f3ab809bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 shutdown_connections 2024-09-13T18:36:40.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.232+0000 7f3abe118640 1 -- 172.21.15.177:0/852791564 wait complete. 2024-09-13T18:36:40.246 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T18:36:40.246 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:40.345 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 Processor -- start 2024-09-13T18:36:40.345 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 -- start start 2024-09-13T18:36:40.345 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406c640 0x7fe27406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.345 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe27406cf60 0x7fe27412a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.345 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe27412ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.345 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27412b090 con 0x7fe27406cf60 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27406c460 con 0x7fe27412a700 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe27a1d5640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe274097e00 con 0x7fe27406c640 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe2737fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406c640 0x7fe27406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe273fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe27412ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe272ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe27406cf60 0x7fe27412a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe273fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe27412ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56800/0 (socket says 172.21.15.177:56800) 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.340+0000 7fe273fff640 1 -- 172.21.15.177:0/87582868 learned_addr learned my addr 172.21.15.177:0/87582868 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2737fe640 1 -- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 msgr2=0x7fe27412ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2737fe640 1 --2- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe27412ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2737fe640 1 -- 172.21.15.177:0/87582868 >> v2:172.21.15.177:3300/0 conn(0x7fe27406cf60 msgr2=0x7fe27412a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2737fe640 1 --2- 172.21.15.177:0/87582868 >> v2:172.21.15.177:3300/0 conn(0x7fe27406cf60 0x7fe27412a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2737fe640 1 -- 172.21.15.177:0/87582868 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27406d770 con 0x7fe27406c640 2024-09-13T18:36:40.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe273fff640 1 --2- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe27412ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe272ffd640 1 --2- 172.21.15.177:0/87582868 >> v2:172.21.15.177:3300/0 conn(0x7fe27406cf60 0x7fe27412a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.350 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2737fe640 1 --2- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406c640 0x7fe27406ca20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe26400ba40 tx=0x7fe26402c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac0951267f751c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.350 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2727fc640 1 -- 172.21.15.177:0/87582868 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264031710 con 0x7fe27406c640 2024-09-13T18:36:40.350 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2727fc640 1 -- 172.21.15.177:0/87582868 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26402f050 con 0x7fe27406c640 2024-09-13T18:36:40.350 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.341+0000 7fe2727fc640 1 -- 172.21.15.177:0/87582868 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264031a90 con 0x7fe27406c640 2024-09-13T18:36:40.350 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 -- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406c640 msgr2=0x7fe27406ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406c640 0x7fe27406ca20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe26400ba40 tx=0x7fe26402c710 comp rx=0 tx=0).stop 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 -- 172.21.15.177:0/87582868 shutdown_connections 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe27412ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/87582868 >> v2:172.21.15.177:3300/0 conn(0x7fe27406cf60 0x7fe27412a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/87582868 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406c640 0x7fe27406ca20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 -- 172.21.15.177:0/87582868 >> 172.21.15.177:0/87582868 conn(0x7fe274098ba0 msgr2=0x7fe274096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 -- 172.21.15.177:0/87582868 shutdown_connections 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.342+0000 7fe27a1d5640 1 -- 172.21.15.177:0/87582868 wait complete. 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 Processor -- start 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 -- start start 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe27406c640 0x7fe2741be130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.351 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 0x7fe2741be670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe2741bebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27408cb90 con 0x7fe27406c640 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe274124710 con 0x7fe27412a700 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.343+0000 7fe27a1d5640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe27408d4a0 con 0x7fe27406cf60 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 0x7fe2741be670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe2737fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe27406c640 0x7fe2741be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe273fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe2741bebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 0x7fe2741be670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60364/0 (socket says 172.21.15.177:60364) 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 -- 172.21.15.177:0/3565409413 learned_addr learned my addr 172.21.15.177:0/3565409413 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 -- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 msgr2=0x7fe2741bebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe2741bebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.352 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 -- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe27406c640 msgr2=0x7fe2741be130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe27406c640 0x7fe2741be130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 -- 172.21.15.177:0/3565409413 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c009400 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe273fff640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe2741bebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe2737fe640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe27406c640 0x7fe2741be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe272ffd640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 0x7fe2741be670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00ec10 tx=0x7fe25c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c009760 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c00cc30 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c013470 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c009b30 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.344+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26400b6a0 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.345+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27406ca20 con 0x7fe27406cf60 2024-09-13T18:36:40.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.345+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fe25c0135d0 con 0x7fe27406cf60 2024-09-13T18:36:40.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.346+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fe25c05cc30 con 0x7fe27406cf60 2024-09-13T18:36:40.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.349+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fe25c03a7a0 con 0x7fe27406cf60 2024-09-13T18:36:40.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.485+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe27412a1c0 con 0x7fe27406cf60 2024-09-13T18:36:40.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.485+0000 7fe26bfff640 1 -- 172.21.15.177:0/3565409413 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+9086 (secure 0 0 0) 0x7fe25c021030 con 0x7fe27406cf60 2024-09-13T18:36:40.487 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:40.488 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":33,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:36:39.660415+0000","last_up_change":"2024-09-13T18:36:35.641745+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":1669150124}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":1669150124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":1669150124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":1669150124}]},"public_addr":"172.21.15.177:6804/1669150124","cluster_addr":"172.21.15.177:6806/1669150124","heartbeat_back_addr":"172.21.15.177:6810/1669150124","heartbeat_front_addr":"172.21.15.177:6809/1669150124","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":2712499331}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":2712499331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":2712499331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":2712499331}]},"public_addr":"172.21.15.177:6805/2712499331","cluster_addr":"172.21.15.177:6807/2712499331","heartbeat_back_addr":"172.21.15.177:6811/2712499331","heartbeat_front_addr":"172.21.15.177:6808/2712499331","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":29,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":341082235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":341082235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":341082235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":341082235}]},"public_addr":"172.21.15.177:6800/341082235","cluster_addr":"172.21.15.177:6801/341082235","heartbeat_back_addr":"172.21.15.177:6803/341082235","heartbeat_front_addr":"172.21.15.177:6802/341082235","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":29,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":2985513301}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":2985513301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":2985513301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":2985513301}]},"public_addr":"172.21.15.177:6812/2985513301","cluster_addr":"172.21.15.177:6813/2985513301","heartbeat_back_addr":"172.21.15.177:6815/2985513301","heartbeat_front_addr":"172.21.15.177:6814/2985513301","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":22},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":23},{"osd":3,"down_stamp":"2024-09-13T18:36:29.602766+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:0/2408340411":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/2735606740":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/3625374812":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/1782828617":"2024-09-14T18:36:22.577532+0000","172.21.15.177:6816/32265":"2024-09-14T18:36:22.577532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:40.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 >> v1:172.21.15.177:6816/516212916 conn(0x7fe254047590 legacy=0x7fe254049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 msgr2=0x7fe2741be670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 0x7fe2741be670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00ec10 tx=0x7fe25c00c520 comp rx=0 tx=0).stop 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 shutdown_connections 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe27412a700 0x7fe2741bebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe27406cf60 0x7fe2741be670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 --2- 172.21.15.177:0/3565409413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe27406c640 0x7fe2741be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 >> 172.21.15.177:0/3565409413 conn(0x7fe274098ba0 msgr2=0x7fe27406ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 shutdown_connections 2024-09-13T18:36:40.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.487+0000 7fe27a1d5640 1 -- 172.21.15.177:0/3565409413 wait complete. 2024-09-13T18:36:40.500 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T18:36:40.501 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T18:36:40.501 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T18:36:40.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T18:36:40.604 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7fd47f308640 1 Processor -- start 2024-09-13T18:36:40.604 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7f98e7ae9640 1 Processor -- start 2024-09-13T18:36:40.605 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7f715f995640 1 Processor -- start 2024-09-13T18:36:40.605 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7fd47f308640 1 -- start start 2024-09-13T18:36:40.605 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7f89faed7640 1 Processor -- start 2024-09-13T18:36:40.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7f98e7ae9640 1 -- start start 2024-09-13T18:36:40.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7f89faed7640 1 -- start start 2024-09-13T18:36:40.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.602+0000 7f715f995640 1 -- start start 2024-09-13T18:36:40.606 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47f308640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 0x7fd478132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e7ae9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006b440 0x7f98e006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e7ae9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f98e006bd60 0x7f98e012d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715f995640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f715812a4d0 0x7f715812a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715f995640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715812adf0 0x7f7158132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.607 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715f995640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715812a2f0 con 0x7f715806e090 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715f995640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7158097e00 con 0x7f715812a4d0 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715f995640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715806e870 con 0x7f715812adf0 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715df32640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715812adf0 0x7f7158132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f715cf30640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f715806e090 0x7f715812f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42626/0 (socket says 172.21.15.177:42626) 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e7ae9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e7ae9640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e006d900 con 0x7f98e006bd60 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e7ae9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e012e3d0 con 0x7f98e006b440 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e7ae9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98e006c4a0 con 0x7f98e012da20 2024-09-13T18:36:40.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e5885640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006b440 0x7f98e006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e5084640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f98e006bd60 0x7f98e012d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e5084640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f98e006bd60 0x7f98e012d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45788/0 (socket says 172.21.15.177:45788) 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e5084640 1 -- 172.21.15.177:0/1326806874 learned_addr learned my addr 172.21.15.177:0/1326806874 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e5885640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006b440 0x7f98e006b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56832/0 (socket says 172.21.15.177:56832) 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f98e6086640 1 --2- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89faed7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89f406cca0 0x7f89f4097b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89faed7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f4095270 0x7f89f4095650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89faed7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f4095fc0 con 0x7f89f4095b90 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89f8c73640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89f406cca0 0x7f89f4097b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89f8c73640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89f406cca0 0x7f89f4097b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45778/0 (socket says 172.21.15.177:45778) 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89f8c73640 1 -- 172.21.15.177:0/1549157571 learned_addr learned my addr 172.21.15.177:0/1549157571 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f406db30 con 0x7f89f406cca0 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89f406e080 con 0x7f89f4095270 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47f308640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd478132d40 0x7fd478135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47c8a3640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 0x7fd478132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47d8a5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd478132d40 0x7fd478135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47c8a3640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 0x7fd478132800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60376/0 (socket says 172.21.15.177:60376) 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47d8a5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd478132d40 0x7fd478135130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56820/0 (socket says 172.21.15.177:56820) 2024-09-13T18:36:40.610 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47d0a4640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd47806b440 0x7fd47806b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42620/0 (socket says 172.21.15.177:42620) 2024-09-13T18:36:40.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47d0a4640 1 -- 172.21.15.177:0/2885104240 learned_addr learned my addr 172.21.15.177:0/2885104240 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:40.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.603+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47806d840 con 0x7fd47806b440 2024-09-13T18:36:40.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47806c4a0 con 0x7fd478132d40 2024-09-13T18:36:40.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd47806db60 con 0x7fd47806bf30 2024-09-13T18:36:40.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f715cf30640 1 -- 172.21.15.177:0/198095986 learned_addr learned my addr 172.21.15.177:0/198095986 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:40.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f89f3fff640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f4095270 0x7f89f4095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f89f8c73640 1 -- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f4095270 msgr2=0x7f89f4095650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f89f8c73640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f4095270 0x7f89f4095650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f89f8c73640 1 -- 172.21.15.177:0/1549157571 >> v1:172.21.15.177:6789/0 conn(0x7f89f4095b90 legacy=0x7f89f406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f715df32640 1 -- 172.21.15.177:0/198095986 >> v2:172.21.15.177:3300/0 conn(0x7f715812a4d0 msgr2=0x7f715812a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f715df32640 1 --2- 172.21.15.177:0/198095986 >> v2:172.21.15.177:3300/0 conn(0x7f715812a4d0 0x7f715812a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f715df32640 1 -- 172.21.15.177:0/198095986 >> v1:172.21.15.177:6789/0 conn(0x7f715806e090 legacy=0x7f715812f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f715df32640 1 -- 172.21.15.177:0/198095986 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715806e9e0 con 0x7f715812adf0 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7fd47c8a3640 1 -- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd478132d40 msgr2=0x7fd478135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7fd47c8a3640 1 --2- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd478132d40 0x7fd478135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7fd47c8a3640 1 -- 172.21.15.177:0/2885104240 >> v1:172.21.15.177:6789/0 conn(0x7fd47806b440 legacy=0x7fd47806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f98e6086640 1 -- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006b440 msgr2=0x7f98e006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f98e6086640 1 --2- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006b440 0x7f98e006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f98e6086640 1 -- 172.21.15.177:0/1326806874 >> v2:172.21.15.177:3300/0 conn(0x7f98e006bd60 msgr2=0x7f98e012d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f98e6086640 1 --2- 172.21.15.177:0/1326806874 >> v2:172.21.15.177:3300/0 conn(0x7f98e006bd60 0x7f98e012d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.613 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7fd47c8a3640 1 -- 172.21.15.177:0/2885104240 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47806cfb0 con 0x7fd47806bf30 2024-09-13T18:36:40.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f98e6086640 1 -- 172.21.15.177:0/1326806874 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e006db60 con 0x7f98e012da20 2024-09-13T18:36:40.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f89f8c73640 1 -- 172.21.15.177:0/1549157571 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f406e390 con 0x7f89f406cca0 2024-09-13T18:36:40.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.604+0000 7f89f3fff640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f4095270 0x7f89f4095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:40.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f715df32640 1 --2- 172.21.15.177:0/198095986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715812adf0 0x7f7158132dc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f71500097a0 tx=0x7f715002a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=903b439004b6971d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f89f8c73640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f406cca0 0x7f89f4097b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009870 tx=0x7f89e802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ffe89ada014985 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f7157fff640 1 -- 172.21.15.177:0/198095986 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715002ad20 con 0x7f715812adf0 2024-09-13T18:36:40.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f89f37fe640 1 -- 172.21.15.177:0/1549157571 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e802ae90 con 0x7f89f406cca0 2024-09-13T18:36:40.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f98e6086640 1 --2- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012de90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009d90 tx=0x7f98d802a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d43e399331c544 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f7157fff640 1 -- 172.21.15.177:0/198095986 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715002ae80 con 0x7f715812adf0 2024-09-13T18:36:40.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f7157fff640 1 -- 172.21.15.177:0/198095986 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150032660 con 0x7f715812adf0 2024-09-13T18:36:40.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f89f37fe640 1 -- 172.21.15.177:0/1549157571 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8032440 con 0x7f89f406cca0 2024-09-13T18:36:40.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f89f37fe640 1 -- 172.21.15.177:0/1549157571 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80327d0 con 0x7f89f406cca0 2024-09-13T18:36:40.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7fd47c8a3640 1 --2- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 0x7fd478132800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd4680097d0 tx=0x7fd46802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53555debc5be464c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f98e4883640 1 -- 172.21.15.177:0/1326806874 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8007740 con 0x7f98e012da20 2024-09-13T18:36:40.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f98e4883640 1 -- 172.21.15.177:0/1326806874 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d802d050 con 0x7f98e012da20 2024-09-13T18:36:40.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.605+0000 7f98e4883640 1 -- 172.21.15.177:0/1326806874 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8007ac0 con 0x7f98e012da20 2024-09-13T18:36:40.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7fd46ffff640 1 -- 172.21.15.177:0/2885104240 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46802ad90 con 0x7fd47806bf30 2024-09-13T18:36:40.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7fd46ffff640 1 -- 172.21.15.177:0/2885104240 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468032440 con 0x7fd47806bf30 2024-09-13T18:36:40.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7fd46ffff640 1 -- 172.21.15.177:0/2885104240 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680327d0 con 0x7fd47806bf30 2024-09-13T18:36:40.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 -- 172.21.15.177:0/198095986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715812adf0 msgr2=0x7f7158132dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 --2- 172.21.15.177:0/198095986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715812adf0 0x7f7158132dc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f71500097a0 tx=0x7f715002a010 comp rx=0 tx=0).stop 2024-09-13T18:36:40.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 -- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 msgr2=0x7f98e012de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 --2- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012de90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009d90 tx=0x7f98d802a490 comp rx=0 tx=0).stop 2024-09-13T18:36:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 -- 172.21.15.177:0/198095986 shutdown_connections 2024-09-13T18:36:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 --2- 172.21.15.177:0/198095986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715812adf0 0x7f7158132dc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 --2- 172.21.15.177:0/198095986 >> v2:172.21.15.177:3300/0 conn(0x7f715812a4d0 0x7f715812a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 -- 172.21.15.177:0/198095986 >> 172.21.15.177:0/198095986 conn(0x7f7158098ba0 msgr2=0x7f7158096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 -- 172.21.15.177:0/1326806874 shutdown_connections 2024-09-13T18:36:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 --2- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012de90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 --2- 172.21.15.177:0/1326806874 >> v2:172.21.15.177:3300/0 conn(0x7f98e006bd60 0x7f98e012d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 --2- 172.21.15.177:0/1326806874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006b440 0x7f98e006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 -- 172.21.15.177:0/1326806874 >> 172.21.15.177:0/1326806874 conn(0x7f98e009fe10 msgr2=0x7f98e00a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f406cca0 msgr2=0x7f89f4097b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f406cca0 0x7f89f4097b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009870 tx=0x7f89e802a140 comp rx=0 tx=0).stop 2024-09-13T18:36:40.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 -- 172.21.15.177:0/1326806874 shutdown_connections 2024-09-13T18:36:40.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f98e7ae9640 1 -- 172.21.15.177:0/1326806874 wait complete. 2024-09-13T18:36:40.621 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 -- 172.21.15.177:0/198095986 shutdown_connections 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 shutdown_connections 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f4095270 0x7f89f4095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 --2- 172.21.15.177:0/1549157571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f406cca0 0x7f89f4097b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 >> 172.21.15.177:0/1549157571 conn(0x7f89f4098ba0 msgr2=0x7f89f4096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 shutdown_connections 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f89faed7640 1 -- 172.21.15.177:0/1549157571 wait complete. 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 msgr2=0x7fd478132800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.622 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 --2- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 0x7fd478132800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd4680097d0 tx=0x7fd46802a040 comp rx=0 tx=0).stop 2024-09-13T18:36:40.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 shutdown_connections 2024-09-13T18:36:40.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 --2- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd478132d40 0x7fd478135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 --2- 172.21.15.177:0/2885104240 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd47806bf30 0x7fd478132800 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 >> 172.21.15.177:0/2885104240 conn(0x7fd47809fe10 msgr2=0x7fd4780a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.606+0000 7f715f995640 1 -- 172.21.15.177:0/198095986 wait complete. 2024-09-13T18:36:40.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 shutdown_connections 2024-09-13T18:36:40.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7fd47f308640 1 -- 172.21.15.177:0/2885104240 wait complete. 2024-09-13T18:36:40.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7f98e7ae9640 1 Processor -- start 2024-09-13T18:36:40.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7f98e7ae9640 1 -- start start 2024-09-13T18:36:40.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7f715f995640 1 Processor -- start 2024-09-13T18:36:40.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.607+0000 7f715f995640 1 -- start start 2024-09-13T18:36:40.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f89faed7640 1 Processor -- start 2024-09-13T18:36:40.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e7ae9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98e006b440 0x7f98e012e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f89faed7640 1 -- start start 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e7ae9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006bd60 0x7f98e012eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e7ae9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e7ae9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00708f0 con 0x7f98e006b440 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e7ae9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e01300d0 con 0x7f98e006bd60 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e7ae9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0130290 con 0x7f98e012da20 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006bd60 0x7f98e012eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e6086640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006bd60 0x7f98e012eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56840/0 (socket says 172.21.15.177:56840) 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 -- 172.21.15.177:0/2808240449 learned_addr learned my addr 172.21.15.177:0/2808240449 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715f995640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715806e090 0x7f7158125fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715f995640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f715812a4d0 0x7f7158126520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715f995640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f715812adf0 0x7f7158126a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715cf30640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f715812a4d0 0x7f7158126520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715cf30640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f715812a4d0 0x7f7158126520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45804/0 (socket says 172.21.15.177:45804) 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7fd47f308640 1 Processor -- start 2024-09-13T18:36:40.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7fd47f308640 1 -- start start 2024-09-13T18:36:40.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 -- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 msgr2=0x7f98e012efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 --2- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f98e012da20 0x7f98e012efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715df32640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f715812adf0 0x7f7158126a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 -- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98e006b440 msgr2=0x7f98e012e560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:36:40.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715df32640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f715812adf0 0x7f7158126a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60418/0 (socket says 172.21.15.177:60418) 2024-09-13T18:36:40.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715cf30640 1 -- 172.21.15.177:0/3747183798 learned_addr learned my addr 172.21.15.177:0/3747183798 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715f995640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715812c8f0 con 0x7f715812a4d0 2024-09-13T18:36:40.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715f995640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7158127a20 con 0x7f715806e090 2024-09-13T18:36:40.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f715f995640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f715812c380 con 0x7f715812adf0 2024-09-13T18:36:40.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5885640 1 --2- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98e006b440 0x7f98e012e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f89faed7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f406cca0 0x7f89f41c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715d731640 1 --2- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715806e090 0x7f7158125fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.608+0000 7f98e5084640 1 --2- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98e006b440 0x7f98e012e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89faed7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f4095270 0x7f89f41c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47f308640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47806b440 0x7fd47812ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f98e5084640 1 -- 172.21.15.177:0/2808240449 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d8009a70 con 0x7f98e006bd60 2024-09-13T18:36:40.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47f308640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd478132d40 0x7fd47812bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47f308640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd47812c490 0x7fd47812c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47d0a4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47806b440 0x7fd47812ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715cf30640 1 -- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f715812adf0 msgr2=0x7f7158126a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89faed7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f4095b90 0x7f89f4093200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89faed7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f4121db0 con 0x7f89f4095b90 2024-09-13T18:36:40.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715cf30640 1 --2- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f715812adf0 0x7f7158126a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715cf30640 1 -- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715806e090 msgr2=0x7f7158125fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715cf30640 1 --2- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f715806e090 0x7f7158125fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89faed7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f4070a90 con 0x7f89f4095270 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89faed7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41225a0 con 0x7f89f406cca0 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89f8c73640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f406cca0 0x7f89f41c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89f8c73640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f406cca0 0x7f89f41c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60430/0 (socket says 172.21.15.177:60430) 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89f8c73640 1 -- 172.21.15.177:0/22069356 learned_addr learned my addr 172.21.15.177:0/22069356 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89f3fff640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f4095270 0x7f89f41c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.634 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47d0a4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47806b440 0x7fd47812ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45820/0 (socket says 172.21.15.177:45820) 2024-09-13T18:36:40.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715cf30640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150009400 con 0x7f715812a4d0 2024-09-13T18:36:40.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f89f9474640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f4095b90 0x7f89f4093200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715df32640 1 --2- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f715812adf0 0x7f7158126a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47d0a4640 1 -- 172.21.15.177:0/1689240482 learned_addr learned my addr 172.21.15.177:0/1689240482 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715cf30640 1 --2- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f715812a4d0 0x7f7158126520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f714400c460 tx=0x7f714400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47812dd20 con 0x7fd47806b440 2024-09-13T18:36:40.636 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd478097870 con 0x7fd47812c490 2024-09-13T18:36:40.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd478127f00 con 0x7fd478132d40 2024-09-13T18:36:40.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144005ce0 con 0x7f715812a4d0 2024-09-13T18:36:40.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f98e5084640 1 --2- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006bd60 0x7f98e012eaa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009fd0 tx=0x7f98dc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7fd47d8a5640 1 --2- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd47812c490 0x7fd47812c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f9474640 1 -- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f406cca0 msgr2=0x7f89f41c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f9474640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f406cca0 0x7f89f41c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f9474640 1 -- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f4095270 msgr2=0x7f89f41c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f9474640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f4095270 0x7f89f41c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f9474640 1 -- 172.21.15.177:0/22069356 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80094d0 con 0x7f89f4095b90 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f8c73640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f89f406cca0 0x7f89f41c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:40.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f3fff640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f4095270 0x7f89f41c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47d0a4640 1 -- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd478132d40 msgr2=0x7fd47812bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47d0a4640 1 --2- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd478132d40 0x7fd47812bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47d0a4640 1 -- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd47812c490 msgr2=0x7fd47812c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47d0a4640 1 --2- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd47812c490 0x7fd47812c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47d0a4640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009400 con 0x7fd47806b440 2024-09-13T18:36:40.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00c8e0 con 0x7f98e006bd60 2024-09-13T18:36:40.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc00ca40 con 0x7f98e006bd60 2024-09-13T18:36:40.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00cdd0 con 0x7f98e006bd60 2024-09-13T18:36:40.640 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f89f9474640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f4095b90 0x7f89f4093200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c690 tx=0x7f89ec00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e01309d0 con 0x7f98e006bd60 2024-09-13T18:36:40.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144005e60 con 0x7f715812a4d0 2024-09-13T18:36:40.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71440072b0 con 0x7f715812a4d0 2024-09-13T18:36:40.641 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.609+0000 7f715f995640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715812b8a0 con 0x7f715812a4d0 2024-09-13T18:36:40.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47d0a4640 1 --2- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47806b440 0x7fd47812ba10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd47400c4a0 tx=0x7fd47400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e01c8910 con 0x7f98e006bd60 2024-09-13T18:36:40.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474005ce0 con 0x7fd47806b440 2024-09-13T18:36:40.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474005e40 con 0x7fd47806b440 2024-09-13T18:36:40.642 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474007270 con 0x7fd47806b440 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.610+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47812d410 con 0x7fd47806b440 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4781d5d00 con 0x7fd47806b440 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98e0134810 con 0x7f98e006bd60 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec003b70 con 0x7f89f4095b90 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec003cd0 con 0x7f89f4095b90 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec003b70 con 0x7f89f4095b90 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f41211f0 con 0x7f89f4095b90 2024-09-13T18:36:40.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f4093740 con 0x7f89f4095b90 2024-09-13T18:36:40.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.611+0000 7f715f995640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71581c2a50 con 0x7f715812a4d0 2024-09-13T18:36:40.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7134000fc0 con 0x7f715812a4d0 2024-09-13T18:36:40.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd47806def0 con 0x7fd47806b440 2024-09-13T18:36:40.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fd474007490 con 0x7fd47806b440 2024-09-13T18:36:40.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f98dc013470 con 0x7f98e006bd60 2024-09-13T18:36:40.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89f4121950 con 0x7f89f4095b90 2024-09-13T18:36:40.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f7144007410 con 0x7f715812a4d0 2024-09-13T18:36:40.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.612+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fd47405be30 con 0x7fd47806b440 2024-09-13T18:36:40.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f98dc05c9d0 con 0x7f98e006bd60 2024-09-13T18:36:40.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f89ec0050d0 con 0x7f89f4095b90 2024-09-13T18:36:40.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 --> v1:172.21.15.177:6812/2985513301 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd45c051aa0 con 0x7fd45c04ddf0 2024-09-13T18:36:40.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd47405e4d0 con 0x7fd47806b440 2024-09-13T18:36:40.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 --> v1:172.21.15.177:6805/2712499331 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98c4051b70 con 0x7f98c404dec0 2024-09-13T18:36:40.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98dc0612a0 con 0x7f98e006bd60 2024-09-13T18:36:40.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f714403b550 con 0x7f715812a4d0 2024-09-13T18:36:40.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f89ec05c090 con 0x7f89f4095b90 2024-09-13T18:36:40.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 --> v1:172.21.15.177:6800/341082235 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89d8051b60 con 0x7f89d804deb0 2024-09-13T18:36:40.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 --> v1:172.21.15.177:6804/1669150124 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f713c051ad0 con 0x7f713c04de20 2024-09-13T18:36:40.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7144039750 con 0x7f715812a4d0 2024-09-13T18:36:40.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.613+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ec003b70 con 0x7f89f4095b90 2024-09-13T18:36:40.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.614+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== osd.3 v1:172.21.15.177:6812/2985513301 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7fd45c051aa0 con 0x7fd45c04ddf0 2024-09-13T18:36:40.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.614+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== osd.1 v1:172.21.15.177:6805/2712499331 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f98c4051b70 con 0x7f98c404dec0 2024-09-13T18:36:40.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.614+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== osd.2 v1:172.21.15.177:6800/341082235 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f89d8051b60 con 0x7f89d804deb0 2024-09-13T18:36:40.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.614+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== osd.0 v1:172.21.15.177:6804/1669150124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f713c051ad0 con 0x7f713c04de20 2024-09-13T18:36:40.649 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.627+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 --> v1:172.21.15.177:6804/1669150124 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7134002bf0 con 0x7f713c04de20 2024-09-13T18:36:40.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.627+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 --> v1:172.21.15.177:6800/341082235 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89f406bb20 con 0x7f89d804deb0 2024-09-13T18:36:40.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f7155ffb640 1 -- 172.21.15.177:0/3747183798 <== osd.0 v1:172.21.15.177:6804/1669150124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 795413372) 0x7f7134002bf0 con 0x7f713c04de20 2024-09-13T18:36:40.650 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89f17fa640 1 -- 172.21.15.177:0/22069356 <== osd.2 v1:172.21.15.177:6800/341082235 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3556540785) 0x7f89e000c020 con 0x7f89d804deb0 2024-09-13T18:36:40.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 >> v1:172.21.15.177:6804/1669150124 conn(0x7f713c04de20 legacy=0x7f713c050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 >> v1:172.21.15.177:6800/341082235 conn(0x7f89d804deb0 legacy=0x7f89d80502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 >> v1:172.21.15.177:6816/516212916 conn(0x7f89d8047620 legacy=0x7f89d8049ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f4095b90 msgr2=0x7f89f4093200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 --2- 172.21.15.177:0/22069356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f4095b90 0x7f89f4093200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c690 tx=0x7f89ec00cb50 comp rx=0 tx=0).stop 2024-09-13T18:36:40.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 >> v1:172.21.15.177:6816/516212916 conn(0x7f713c047590 legacy=0x7f713c049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89f9474640 1 -- 172.21.15.177:0/22069356 reap_dead start 2024-09-13T18:36:40.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f715812a4d0 msgr2=0x7f7158126520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 --2- 172.21.15.177:0/3747183798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f715812a4d0 0x7f7158126520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f714400c460 tx=0x7f714400c920 comp rx=0 tx=0).stop 2024-09-13T18:36:40.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f715df32640 1 -- 172.21.15.177:0/3747183798 reap_dead start 2024-09-13T18:36:40.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 shutdown_connections 2024-09-13T18:36:40.654 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 >> 172.21.15.177:0/3747183798 conn(0x7f7158098ba0 msgr2=0x7f7158127f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 shutdown_connections 2024-09-13T18:36:40.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 >> 172.21.15.177:0/22069356 conn(0x7f89f4098ba0 msgr2=0x7f89f406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 shutdown_connections 2024-09-13T18:36:40.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f89faed7640 1 -- 172.21.15.177:0/22069356 wait complete. 2024-09-13T18:36:40.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.628+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 shutdown_connections 2024-09-13T18:36:40.655 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.629+0000 7f713b7fe640 1 -- 172.21.15.177:0/3747183798 wait complete. 2024-09-13T18:36:40.656 INFO:teuthology.orchestra.run.smithi177.stdout:120259084291 2024-09-13T18:36:40.657 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:40.657 INFO:teuthology.orchestra.run.smithi177.stdout:111669149699 2024-09-13T18:36:40.658 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:40.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.644+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 --> v1:172.21.15.177:6805/2712499331 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98e01316c0 con 0x7f98c404dec0 2024-09-13T18:36:40.658 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.644+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 --> v1:172.21.15.177:6812/2985513301 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd478135ac0 con 0x7fd45c04ddf0 2024-09-13T18:36:40.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.644+0000 7f98d67fc640 1 -- 172.21.15.177:0/2808240449 <== osd.1 v1:172.21.15.177:6805/2712499331 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3324758384) 0x7f98d8003990 con 0x7f98c404dec0 2024-09-13T18:36:40.659 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd46dffb640 1 -- 172.21.15.177:0/1689240482 <== osd.3 v1:172.21.15.177:6812/2985513301 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1318614166) 0x7fd478135ac0 con 0x7fd45c04ddf0 2024-09-13T18:36:40.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 >> v1:172.21.15.177:6805/2712499331 conn(0x7f98c404dec0 legacy=0x7f98c4050300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 >> v1:172.21.15.177:6816/516212916 conn(0x7f98c4047630 legacy=0x7f98c4049af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006bd60 msgr2=0x7f98e012eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.660 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 --2- 172.21.15.177:0/2808240449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98e006bd60 0x7f98e012eaa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009fd0 tx=0x7f98dc00ed40 comp rx=0 tx=0).stop 2024-09-13T18:36:40.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e6086640 1 -- 172.21.15.177:0/2808240449 reap_dead start 2024-09-13T18:36:40.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 shutdown_connections 2024-09-13T18:36:40.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 >> 172.21.15.177:0/2808240449 conn(0x7f98e009fe10 msgr2=0x7f98e012a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 >> v1:172.21.15.177:6812/2985513301 conn(0x7fd45c04ddf0 legacy=0x7fd45c050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 shutdown_connections 2024-09-13T18:36:40.661 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7f98e7ae9640 1 -- 172.21.15.177:0/2808240449 wait complete. 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 >> v1:172.21.15.177:6816/516212916 conn(0x7fd45c047560 legacy=0x7fd45c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47806b440 msgr2=0x7fd47812ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 --2- 172.21.15.177:0/1689240482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47806b440 0x7fd47812ba10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd47400c4a0 tx=0x7fd47400c960 comp rx=0 tx=0).stop 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47d8a5640 1 -- 172.21.15.177:0/1689240482 reap_dead start 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 shutdown_connections 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 >> 172.21.15.177:0/1689240482 conn(0x7fd47809fe10 msgr2=0x7fd47809bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 shutdown_connections 2024-09-13T18:36:40.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.645+0000 7fd47f308640 1 -- 172.21.15.177:0/1689240482 wait complete. 2024-09-13T18:36:40.664 INFO:teuthology.orchestra.run.smithi177.stdout:124554051587 2024-09-13T18:36:40.664 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:40.665 INFO:teuthology.orchestra.run.smithi177.stdout:115964116995 2024-09-13T18:36:40.665 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:40.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 Processor -- start 2024-09-13T18:36:40.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 -- start start 2024-09-13T18:36:40.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f323c06b440 0x7f323c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 0x7f323c12d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c12da20 0x7f323c12deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c06d840 con 0x7f323c06b440 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c06c4a0 con 0x7f323c06bd60 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.763+0000 7f32427e9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f323c06db60 con 0x7f323c12da20 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 0x7f323c12d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 0x7f323c12d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56880/0 (socket says 172.21.15.177:56880) 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 -- 172.21.15.177:0/4063628212 learned_addr learned my addr 172.21.15.177:0/4063628212 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323bfff640 1 --2- 172.21.15.177:0/4063628212 >> v2:172.21.15.177:3300/0 conn(0x7f323c06b440 0x7f323c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f3240d86640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c12da20 0x7f323c12deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 -- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c12da20 msgr2=0x7f323c12deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c12da20 0x7f323c12deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 -- 172.21.15.177:0/4063628212 >> v2:172.21.15.177:3300/0 conn(0x7f323c06b440 msgr2=0x7f323c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.768 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 --2- 172.21.15.177:0/4063628212 >> v2:172.21.15.177:3300/0 conn(0x7f323c06b440 0x7f323c06b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.768 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323b7fe640 1 -- 172.21.15.177:0/4063628212 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c06c170 con 0x7f323c06bd60 2024-09-13T18:36:40.768 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f3240d86640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c12da20 0x7f323c12deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:40.768 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.764+0000 7f323bfff640 1 --2- 172.21.15.177:0/4063628212 >> v2:172.21.15.177:3300/0 conn(0x7f323c06b440 0x7f323c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:40.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.765+0000 7f323b7fe640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 0x7f323c12d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3230009870 tx=0x7f323002a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38da9535498a3f95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.765+0000 7f323affd640 1 -- 172.21.15.177:0/4063628212 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323002adf0 con 0x7f323c06bd60 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.765+0000 7f323affd640 1 -- 172.21.15.177:0/4063628212 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3230032440 con 0x7f323c06bd60 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.765+0000 7f323affd640 1 -- 172.21.15.177:0/4063628212 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32300327d0 con 0x7f323c06bd60 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 -- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 msgr2=0x7f323c12d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 0x7f323c12d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3230009870 tx=0x7f323002a0a0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 -- 172.21.15.177:0/4063628212 shutdown_connections 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c12da20 0x7f323c12deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 --2- 172.21.15.177:0/4063628212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c06bd60 0x7f323c12d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 --2- 172.21.15.177:0/4063628212 >> v2:172.21.15.177:3300/0 conn(0x7f323c06b440 0x7f323c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 -- 172.21.15.177:0/4063628212 >> 172.21.15.177:0/4063628212 conn(0x7f323c09fe10 msgr2=0x7f323c0a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 -- 172.21.15.177:0/4063628212 shutdown_connections 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.766+0000 7f32427e9640 1 -- 172.21.15.177:0/4063628212 wait complete. 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 Processor -- start 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 -- start start 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c06b440 0x7f323c12e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f323c06bd60 0x7f323c12ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 0x7f323c12efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1317c0 con 0x7f323c06bd60 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c127eb0 con 0x7f323c12da20 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f32427e9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f323c070c60 con 0x7f323c06b440 2024-09-13T18:36:40.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f3240d86640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 0x7f323c12efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f3240d86640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 0x7f323c12efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56894/0 (socket says 172.21.15.177:56894) 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f3240d86640 1 -- 172.21.15.177:0/209225570 learned_addr learned my addr 172.21.15.177:0/209225570 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f323b7fe640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f323c06bd60 0x7f323c12ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.767+0000 7f323bfff640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c06b440 0x7f323c12e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323bfff640 1 -- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 msgr2=0x7f323c12efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323bfff640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 0x7f323c12efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323bfff640 1 -- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f323c06bd60 msgr2=0x7f323c12ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323bfff640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f323c06bd60 0x7f323c12ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323bfff640 1 -- 172.21.15.177:0/209225570 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300094d0 con 0x7f323c06b440 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f3240d86640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 0x7f323c12efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323b7fe640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f323c06bd60 0x7f323c12ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f323bfff640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c06b440 0x7f323c12e540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f322c00e5e0 tx=0x7f322c00eaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c004230 con 0x7f323c06b440 2024-09-13T18:36:40.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c130ef0 con 0x7f323c06b440 2024-09-13T18:36:40.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c004390 con 0x7f323c06b440 2024-09-13T18:36:40.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c003850 con 0x7f323c06b440 2024-09-13T18:36:40.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.768+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c1c8a60 con 0x7f323c06b440 2024-09-13T18:36:40.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.770+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c12fb80 con 0x7f323c06b440 2024-09-13T18:36:40.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.770+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f322c0039b0 con 0x7f323c06b440 2024-09-13T18:36:40.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.770+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f322c05c7f0 con 0x7f323c06b440 2024-09-13T18:36:40.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.777+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f322c03a3c0 con 0x7f323c06b440 2024-09-13T18:36:40.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.805+0000 7fe44f7b5640 1 Processor -- start 2024-09-13T18:36:40.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.805+0000 7f1636dd4640 1 Processor -- start 2024-09-13T18:36:40.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.805+0000 7fe44f7b5640 1 -- start start 2024-09-13T18:36:40.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.805+0000 7f1636dd4640 1 -- start start 2024-09-13T18:36:40.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44f7b5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe44806b440 0x7fe44806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1636dd4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f1630132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44f7b5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44812dbf0 0x7fe44812e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1636dd4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1630132d40 0x7f1630135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44f7b5640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44806c4a0 con 0x7fe44806bd60 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44f7b5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44806d900 con 0x7fe44806b440 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44f7b5640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe44806db60 con 0x7fe44812dbf0 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1636dd4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163006d840 con 0x7f163006b440 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1636dd4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163006c4a0 con 0x7f163006bf30 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1636dd4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f163006db60 con 0x7f1630132d40 2024-09-13T18:36:40.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1634b70640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f163006b440 0x7f163006b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42654/0 (socket says 172.21.15.177:42654) 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1634b70640 1 -- 172.21.15.177:0/4174038833 learned_addr learned my addr 172.21.15.177:0/4174038833 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f162ffff640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f1630132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44cd50640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe44806bd60 0x7fe44812d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42638/0 (socket says 172.21.15.177:42638) 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44cd50640 1 -- 172.21.15.177:0/2026390699 learned_addr learned my addr 172.21.15.177:0/2026390699 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44d551640 1 --2- 172.21.15.177:0/2026390699 >> v2:172.21.15.177:3300/0 conn(0x7fe44806b440 0x7fe44806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7fe44dd52640 1 --2- 172.21.15.177:0/2026390699 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44812dbf0 0x7fe44812e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.811 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.806+0000 7f1635371640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1630132d40 0x7f1630135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.812 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7fe43ffff640 1 -- 172.21.15.177:0/2026390699 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313644867 0 0) 0x7fe44806c4a0 con 0x7fe44806bd60 2024-09-13T18:36:40.812 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7fe44dd52640 1 -- 172.21.15.177:0/2026390699 >> v2:172.21.15.177:3300/0 conn(0x7fe44806b440 msgr2=0x7fe44806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.812 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7fe44dd52640 1 --2- 172.21.15.177:0/2026390699 >> v2:172.21.15.177:3300/0 conn(0x7fe44806b440 0x7fe44806b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.812 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7fe44dd52640 1 -- 172.21.15.177:0/2026390699 >> v1:172.21.15.177:6789/0 conn(0x7fe44806bd60 legacy=0x7fe44812d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.812 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7fe44dd52640 1 -- 172.21.15.177:0/2026390699 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44806c170 con 0x7fe44812dbf0 2024-09-13T18:36:40.812 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7fe44dd52640 1 --2- 172.21.15.177:0/2026390699 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44812dbf0 0x7fe44812e040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe440009dc0 tx=0x7fe44002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157424ab22567216 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7f162ffff640 1 -- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1630132d40 msgr2=0x7f1630135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7f162ffff640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1630132d40 0x7f1630135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7f162ffff640 1 -- 172.21.15.177:0/4174038833 >> v1:172.21.15.177:6789/0 conn(0x7f163006b440 legacy=0x7f163006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7f162ffff640 1 -- 172.21.15.177:0/4174038833 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163006cfb0 con 0x7f163006bf30 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.807+0000 7f1635371640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1630132d40 0x7f1630135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7fe43ffff640 1 -- 172.21.15.177:0/2026390699 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440007740 con 0x7fe44812dbf0 2024-09-13T18:36:40.813 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 Processor -- start 2024-09-13T18:36:40.814 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7fe43ffff640 1 -- 172.21.15.177:0/2026390699 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44002d050 con 0x7fe44812dbf0 2024-09-13T18:36:40.814 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7fe43ffff640 1 -- 172.21.15.177:0/2026390699 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440007ac0 con 0x7fe44812dbf0 2024-09-13T18:36:40.814 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 -- start start 2024-09-13T18:36:40.815 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f162ffff640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f1630132800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f161c0097d0 tx=0x7f161c02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b672e97f0ab6440c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.815 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f162f7fe640 1 -- 172.21.15.177:0/4174038833 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c02ad90 con 0x7f163006bf30 2024-09-13T18:36:40.815 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f162f7fe640 1 -- 172.21.15.177:0/4174038833 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c032440 con 0x7f163006bf30 2024-09-13T18:36:40.815 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f162f7fe640 1 -- 172.21.15.177:0/4174038833 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c0327d0 con 0x7f163006bf30 2024-09-13T18:36:40.815 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f912012bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.815 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f912006f080 0x7f91201336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912012b510 con 0x7f912006e530 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 -- 172.21.15.177:0/2026390699 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44812dbf0 msgr2=0x7fe44812e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/2026390699 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44812dbf0 0x7fe44812e040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe440009dc0 tx=0x7fe44002a560 comp rx=0 tx=0).stop 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912006b480 con 0x7f912006f080 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9125074640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f912006f080 0x7f91201336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.808+0000 7f9126ad7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f912006ff40 con 0x7f912012b920 2024-09-13T18:36:40.816 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9125074640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f912006f080 0x7f91201336d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45870/0 (socket says 172.21.15.177:45870) 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9125074640 1 -- 172.21.15.177:0/2820967063 learned_addr learned my addr 172.21.15.177:0/2820967063 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9117fff640 1 --1- 172.21.15.177:0/2820967063 >> v1:172.21.15.177:6789/0 conn(0x7f912006e530 0x7f912006e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42662/0 (socket says 172.21.15.177:42662) 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9124873640 1 --2- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f912012bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 -- 172.21.15.177:0/2026390699 shutdown_connections 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/2026390699 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44812dbf0 0x7fe44812e040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/2026390699 >> v2:172.21.15.177:3300/0 conn(0x7fe44806b440 0x7fe44806b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 -- 172.21.15.177:0/2026390699 >> 172.21.15.177:0/2026390699 conn(0x7fe44809fe10 msgr2=0x7fe4480a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 -- 172.21.15.177:0/2026390699 shutdown_connections 2024-09-13T18:36:40.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7fe44f7b5640 1 -- 172.21.15.177:0/2026390699 wait complete. 2024-09-13T18:36:40.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9125074640 1 -- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 msgr2=0x7f912012bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:40.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9125074640 1 --2- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f912012bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9125074640 1 -- 172.21.15.177:0/2820967063 >> v1:172.21.15.177:6789/0 conn(0x7f912006e530 legacy=0x7f912006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f9125074640 1 -- 172.21.15.177:0/2820967063 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912012c240 con 0x7f912006f080 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 -- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 msgr2=0x7f1630132800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f1630132800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f161c0097d0 tx=0x7f161c02a040 comp rx=0 tx=0).stop 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 -- 172.21.15.177:0/4174038833 shutdown_connections 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f1630132800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 --2- 172.21.15.177:0/4174038833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1630132d40 0x7f1630135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 -- 172.21.15.177:0/4174038833 >> 172.21.15.177:0/4174038833 conn(0x7f163009fe10 msgr2=0x7f16300a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 -- 172.21.15.177:0/4174038833 shutdown_connections 2024-09-13T18:36:40.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.809+0000 7f1636dd4640 1 -- 172.21.15.177:0/4174038833 wait complete. 2024-09-13T18:36:40.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7f9125074640 1 --2- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201336d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f911800dc90 tx=0x7f911802e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5d5d5ccfc715cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7f91177fe640 1 -- 172.21.15.177:0/2820967063 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800b650 con 0x7f912006f080 2024-09-13T18:36:40.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7fe44f7b5640 1 Processor -- start 2024-09-13T18:36:40.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7f91177fe640 1 -- 172.21.15.177:0/2820967063 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118031050 con 0x7f912006f080 2024-09-13T18:36:40.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7f91177fe640 1 -- 172.21.15.177:0/2820967063 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800b9d0 con 0x7f912006f080 2024-09-13T18:36:40.821 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7fe44f7b5640 1 -- start start 2024-09-13T18:36:40.821 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7f1636dd4640 1 Processor -- start 2024-09-13T18:36:40.821 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7fe44f7b5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 0x7fe4480999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.821 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7fe44f7b5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44806bd60 0x7fe448099f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.822 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.810+0000 7f1636dd4640 1 -- start start 2024-09-13T18:36:40.822 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44f7b5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44812dbf0 0x7fe448095680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.822 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 0x7fe4480999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44f7b5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448070460 con 0x7fe44806b440 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44f7b5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe448097d00 con 0x7fe44812dbf0 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44f7b5640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe448096f80 con 0x7fe44806bd60 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 0x7fe4480999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45880/0 (socket says 172.21.15.177:45880) 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 -- 172.21.15.177:0/3124288869 learned_addr learned my addr 172.21.15.177:0/3124288869 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44cd50640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44806bd60 0x7fe448099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44dd52640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44812dbf0 0x7fe448095680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 -- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 msgr2=0x7f91201336d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.823 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 --2- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201336d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f911800dc90 tx=0x7f911802e440 comp rx=0 tx=0).stop 2024-09-13T18:36:40.824 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 -- 172.21.15.177:0/2820967063 shutdown_connections 2024-09-13T18:36:40.824 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 -- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44806bd60 msgr2=0x7fe448099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.824 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44806bd60 0x7fe448099f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.824 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 -- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44812dbf0 msgr2=0x7fe448095680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.824 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44812dbf0 0x7fe448095680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.824 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 -- 172.21.15.177:0/3124288869 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009a70 con 0x7fe44806b440 2024-09-13T18:36:40.825 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 --2- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201336d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.825 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 --2- 172.21.15.177:0/2820967063 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f912012bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.825 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 -- 172.21.15.177:0/2820967063 >> 172.21.15.177:0/2820967063 conn(0x7f912009ccd0 msgr2=0x7f912009d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.825 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 -- 172.21.15.177:0/2820967063 shutdown_connections 2024-09-13T18:36:40.825 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f9126ad7640 1 -- 172.21.15.177:0/2820967063 wait complete. 2024-09-13T18:36:40.825 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1636dd4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 0x7f16301c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7fe44d551640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 0x7fe4480999d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe44400e9b0 tx=0x7fe44400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1636dd4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f16301c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1636dd4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1630132d40 0x7f163006e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1634b70640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 0x7f16301c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1636dd4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163006f8f0 con 0x7f1630132d40 2024-09-13T18:36:40.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1636dd4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163006fb20 con 0x7f163006bf30 2024-09-13T18:36:40.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1636dd4640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16301c3810 con 0x7f163006b440 2024-09-13T18:36:40.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444009670 con 0x7fe44806b440 2024-09-13T18:36:40.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f162ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f16301c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1634b70640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 0x7f16301c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60506/0 (socket says 172.21.15.177:60506) 2024-09-13T18:36:40.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.811+0000 7f1634b70640 1 -- 172.21.15.177:0/517810285 learned_addr learned my addr 172.21.15.177:0/517810285 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44400c9a0 con 0x7fe44806b440 2024-09-13T18:36:40.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe448127eb0 con 0x7fe44806b440 2024-09-13T18:36:40.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1634b70640 1 -- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 msgr2=0x7f16301c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1634b70640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f16301c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1634b70640 1 -- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1630132d40 msgr2=0x7f163006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1635371640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1630132d40 0x7f163006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1634b70640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1630132d40 0x7f163006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1634b70640 1 -- 172.21.15.177:0/517810285 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1628009a70 con 0x7f163006b440 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f162ffff640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f16301c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1635371640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1630132d40 0x7f163006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:40.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44400cd50 con 0x7fe44806b440 2024-09-13T18:36:40.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1634b70640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 0x7f16301c2d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f162400c4a0 tx=0x7f162400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f9126ad7640 1 Processor -- start 2024-09-13T18:36:40.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624005ce0 con 0x7f163006b440 2024-09-13T18:36:40.831 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c009400 con 0x7f163006b440 2024-09-13T18:36:40.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624005e40 con 0x7f163006b440 2024-09-13T18:36:40.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624007270 con 0x7f163006b440 2024-09-13T18:36:40.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe448097460 con 0x7fe44806b440 2024-09-13T18:36:40.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1630070a70 con 0x7f163006b440 2024-09-13T18:36:40.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.812+0000 7f9126ad7640 1 -- start start 2024-09-13T18:36:40.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7f9126ad7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 0x7f91201c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7f9126ad7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7f9126ad7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f91201c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:40.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7f9117fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.833 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1624007490 con 0x7f163006b440 2024-09-13T18:36:40.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9117fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45886/0 (socket says 172.21.15.177:45886) 2024-09-13T18:36:40.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9124873640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 0x7f91201c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f162405bf30 con 0x7f163006b440 2024-09-13T18:36:40.834 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630070c60 con 0x7f163006b440 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9124873640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 0x7f91201c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56936/0 (socket says 172.21.15.177:56936) 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9125074640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f91201c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9117fff640 1 -- 172.21.15.177:0/1130627390 learned_addr learned my addr 172.21.15.177:0/1130627390 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9120094fb0 con 0x7f912006f080 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9120124a80 con 0x7f912006e530 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9120095180 con 0x7f912012b920 2024-09-13T18:36:40.835 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7f9125074640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f91201c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60508/0 (socket says 172.21.15.177:60508) 2024-09-13T18:36:40.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.813+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fe444013470 con 0x7fe44806b440 2024-09-13T18:36:40.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.814+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fe44405cb20 con 0x7fe44806b440 2024-09-13T18:36:40.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9124873640 1 -- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 msgr2=0x7f91201c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9124873640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f91201c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9124873640 1 -- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 msgr2=0x7f91201c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9124873640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9124873640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911800d970 con 0x7f912006e530 2024-09-13T18:36:40.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9124873640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 0x7f91201c6770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f911c00efc0 tx=0x7f911c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:40.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe448097650 con 0x7fe44806b440 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c014480 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9120095720 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91200939f0 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c00f040 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.815+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c014480 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.817+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9100005250 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.817+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f911c0145e0 con 0x7f912006e530 2024-09-13T18:36:40.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.817+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f911c03cd50 con 0x7f912006e530 2024-09-13T18:36:40.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.820+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f1624039a80 con 0x7f163006b440 2024-09-13T18:36:40.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.823+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fe444017080 con 0x7fe44806b440 2024-09-13T18:36:40.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.823+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f911c03b6e0 con 0x7f912006e530 2024-09-13T18:36:40.930 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.928+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f323c12c460 con 0x7f323c06b440 2024-09-13T18:36:40.930 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.928+0000 7f3238ff9640 1 -- 172.21.15.177:0/209225570 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f322c014070 con 0x7f323c06b440 2024-09-13T18:36:40.930 INFO:teuthology.orchestra.run.smithi177.stdout:120259084290 2024-09-13T18:36:40.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.930+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 >> v1:172.21.15.177:6816/516212916 conn(0x7f3228047560 legacy=0x7f3228049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.930+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c06b440 msgr2=0x7f323c12e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.930+0000 7f32427e9640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c06b440 0x7f323c12e540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f322c00e5e0 tx=0x7f322c00eaa0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 shutdown_connections 2024-09-13T18:36:40.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f323c12da20 0x7f323c12efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f323c06bd60 0x7f323c12ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.934 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 --2- 172.21.15.177:0/209225570 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f323c06b440 0x7f323c12e540 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.934 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 >> 172.21.15.177:0/209225570 conn(0x7f323c09fe10 msgr2=0x7f323c09bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.934 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 shutdown_connections 2024-09-13T18:36:40.934 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.931+0000 7f32427e9640 1 -- 172.21.15.177:0/209225570 wait complete. 2024-09-13T18:36:40.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084291 got 120259084290 for osd.2 2024-09-13T18:36:40.967 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.965+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f163009d490 con 0x7f163006b440 2024-09-13T18:36:40.967 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.965+0000 7f162d7fa640 1 -- 172.21.15.177:0/517810285 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1624017030 con 0x7f163006b440 2024-09-13T18:36:40.967 INFO:teuthology.orchestra.run.smithi177.stdout:111669149698 2024-09-13T18:36:40.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.966+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe448073cc0 con 0x7fe44806b440 2024-09-13T18:36:40.968 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.966+0000 7fe43dffb640 1 -- 172.21.15.177:0/3124288869 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe44403a710 con 0x7fe44806b440 2024-09-13T18:36:40.969 INFO:teuthology.orchestra.run.smithi177.stdout:124554051586 2024-09-13T18:36:40.969 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 >> v1:172.21.15.177:6816/516212916 conn(0x7f1614047560 legacy=0x7f1614049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.969 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 msgr2=0x7f16301c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 0x7f16301c2d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f162400c4a0 tx=0x7f162400c960 comp rx=0 tx=0).stop 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 shutdown_connections 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1630132d40 0x7f163006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f163006bf30 0x7f16301c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 --2- 172.21.15.177:0/517810285 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f163006b440 0x7f16301c2d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 >> 172.21.15.177:0/517810285 conn(0x7f163009fe10 msgr2=0x7f1630127420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 shutdown_connections 2024-09-13T18:36:40.970 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.967+0000 7f1636dd4640 1 -- 172.21.15.177:0/517810285 wait complete. 2024-09-13T18:36:40.971 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 >> v1:172.21.15.177:6816/516212916 conn(0x7fe42c047660 legacy=0x7fe42c049b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 msgr2=0x7fe4480999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 0x7fe4480999d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe44400e9b0 tx=0x7fe44400ee70 comp rx=0 tx=0).stop 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 shutdown_connections 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44812dbf0 0x7fe448095680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fe44806bd60 0x7fe448099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 --2- 172.21.15.177:0/3124288869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44806b440 0x7fe4480999d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 >> 172.21.15.177:0/3124288869 conn(0x7fe44809fe10 msgr2=0x7fe44812c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 shutdown_connections 2024-09-13T18:36:40.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.968+0000 7fe44f7b5640 1 -- 172.21.15.177:0/3124288869 wait complete. 2024-09-13T18:36:40.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.975+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f91000057d0 con 0x7f912006e530 2024-09-13T18:36:40.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.975+0000 7f91157fa640 1 -- 172.21.15.177:0/1130627390 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f911c03b6e0 con 0x7f912006e530 2024-09-13T18:36:40.977 INFO:teuthology.orchestra.run.smithi177.stdout:115964116994 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 >> v1:172.21.15.177:6816/516212916 conn(0x7f910c047620 legacy=0x7f910c049ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 msgr2=0x7f91201c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 0x7f91201c6770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f911c00efc0 tx=0x7f911c00c490 comp rx=0 tx=0).stop 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 shutdown_connections 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f912012b920 0x7f91201c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912006f080 0x7f91201c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 --2- 172.21.15.177:0/1130627390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f912006e530 0x7f91201c6770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:40.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 >> 172.21.15.177:0/1130627390 conn(0x7f912009ccd0 msgr2=0x7f9120097e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:40.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 shutdown_connections 2024-09-13T18:36:40.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:40.977+0000 7f9126ad7640 1 -- 172.21.15.177:0/1130627390 wait complete. 2024-09-13T18:36:40.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 111669149699 got 111669149698 for osd.0 2024-09-13T18:36:40.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051587 got 124554051586 for osd.3 2024-09-13T18:36:40.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.1 2024-09-13T18:36:41.949 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:41.983 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:41.983 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:41.991 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:42.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.062+0000 7f4b89ee9640 1 Processor -- start 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.062+0000 7f4b89ee9640 1 -- start start 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b89ee9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 0x7f4b8406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b89ee9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4b8406bd60 0x7f4b8412d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b89ee9640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8406c4a0 con 0x7f4b8412da20 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b89ee9640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8406d8d0 con 0x7f4b8406bd60 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b89ee9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8406db60 con 0x7f4b8406b440 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 0x7f4b8406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 0x7f4b8406b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60514/0 (socket says 172.21.15.177:60514) 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b837fe640 1 -- 172.21.15.177:0/4137956123 learned_addr learned my addr 172.21.15.177:0/4137956123 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b82ffd640 1 --2- 172.21.15.177:0/4137956123 >> v2:172.21.15.177:3300/0 conn(0x7f4b8406bd60 0x7f4b8412d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b82ffd640 1 -- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 msgr2=0x7f4b8406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b82ffd640 1 --2- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 0x7f4b8406b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.063+0000 7f4b827fc640 1 -- 172.21.15.177:0/4137956123 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190882741 0 0) 0x7f4b8406c4a0 con 0x7f4b8412da20 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b82ffd640 1 -- 172.21.15.177:0/4137956123 >> v1:172.21.15.177:6789/0 conn(0x7f4b8412da20 legacy=0x7f4b8412de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b82ffd640 1 -- 172.21.15.177:0/4137956123 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8406c140 con 0x7f4b8406bd60 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b837fe640 1 --2- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 0x7f4b8406b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b82ffd640 1 --2- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8406bd60 0x7f4b8412d4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b780097a0 tx=0x7f4b7802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314be40a95a20066 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b827fc640 1 -- 172.21.15.177:0/4137956123 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7802adb0 con 0x7f4b8406bd60 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b827fc640 1 -- 172.21.15.177:0/4137956123 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78032440 con 0x7f4b8406bd60 2024-09-13T18:36:42.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.064+0000 7f4b827fc640 1 -- 172.21.15.177:0/4137956123 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780327d0 con 0x7f4b8406bd60 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 -- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8406bd60 msgr2=0x7f4b8412d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8406bd60 0x7f4b8412d4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b780097a0 tx=0x7f4b7802a060 comp rx=0 tx=0).stop 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 -- 172.21.15.177:0/4137956123 shutdown_connections 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8406bd60 0x7f4b8412d4e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/4137956123 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406b440 0x7f4b8406b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 -- 172.21.15.177:0/4137956123 >> 172.21.15.177:0/4137956123 conn(0x7f4b8409fe10 msgr2=0x7f4b840a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 -- 172.21.15.177:0/4137956123 shutdown_connections 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.065+0000 7f4b89ee9640 1 -- 172.21.15.177:0/4137956123 wait complete. 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.067+0000 7f4b89ee9640 1 Processor -- start 2024-09-13T18:36:42.070 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.067+0000 7f4b89ee9640 1 -- start start 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.067+0000 7f4b89ee9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b8406b440 0x7f4b8412e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.067+0000 7f4b89ee9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 0x7f4b8412eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.067+0000 7f4b89ee9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 0x7f4b8412f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b8406b440 0x7f4b8412e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b8406b440 0x7f4b8412e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56938/0 (socket says 172.21.15.177:56938) 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b82ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 0x7f4b8412eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 0x7f4b8412f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b82ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 0x7f4b8412eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60530/0 (socket says 172.21.15.177:60530) 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 0x7f4b8412f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45902/0 (socket says 172.21.15.177:45902) 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b837fe640 1 -- 172.21.15.177:0/1033072097 learned_addr learned my addr 172.21.15.177:0/1033072097 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84131090 con 0x7f4b8412da20 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84131460 con 0x7f4b8406b440 2024-09-13T18:36:42.071 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84130020 con 0x7f4b8406bd60 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 -- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 msgr2=0x7f4b8412eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 0x7f4b8412eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 -- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b8406b440 msgr2=0x7f4b8412e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b8406b440 0x7f4b8412e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b83fff640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740094d0 con 0x7f4b8412da20 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.068+0000 7f4b82ffd640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 0x7f4b8412eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.069+0000 7f4b83fff640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 0x7f4b8412f030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c001560 tx=0x7f4b7c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.069+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003da0 con 0x7f4b8412da20 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.069+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00ce30 con 0x7f4b8412da20 2024-09-13T18:36:42.073 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.069+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b78009400 con 0x7f4b8412da20 2024-09-13T18:36:42.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.069+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c005290 con 0x7f4b8412da20 2024-09-13T18:36:42.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.069+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b840706f0 con 0x7f4b8412da20 2024-09-13T18:36:42.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.070+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b84130b90 con 0x7f4b8412da20 2024-09-13T18:36:42.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.071+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f4b7c0053f0 con 0x7f4b8412da20 2024-09-13T18:36:42.074 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.071+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f4b7c05c140 con 0x7f4b8412da20 2024-09-13T18:36:42.081 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.078+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f4b7c039db0 con 0x7f4b8412da20 2024-09-13T18:36:42.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.101+0000 7fd9474f0640 1 Processor -- start 2024-09-13T18:36:42.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.101+0000 7fd9474f0640 1 -- start start 2024-09-13T18:36:42.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.101+0000 7fcffae6b640 1 Processor -- start 2024-09-13T18:36:42.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcffae6b640 1 -- start start 2024-09-13T18:36:42.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd9474f0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd94006b440 0x7fd94006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd9474f0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94012dbf0 0x7fd94012e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd9474f0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94006c4a0 con 0x7fd94006bd60 2024-09-13T18:36:42.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcffae6b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcff406c590 0x7fcff406c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd944a8b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd94006bd60 0x7fd94012d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42684/0 (socket says 172.21.15.177:42684) 2024-09-13T18:36:42.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd944a8b640 1 -- 172.21.15.177:0/2564711309 learned_addr learned my addr 172.21.15.177:0/2564711309 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:42.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd94528c640 1 --2- 172.21.15.177:0/2564711309 >> v2:172.21.15.177:3300/0 conn(0x7fd94006b440 0x7fd94006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94006d900 con 0x7fd94006b440 2024-09-13T18:36:42.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd945a8d640 1 --2- 172.21.15.177:0/2564711309 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94012dbf0 0x7fd94012e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd94006db60 con 0x7fd94012dbf0 2024-09-13T18:36:42.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcffae6b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff406ceb0 0x7fcff412a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcff8c07640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcff406c590 0x7fcff406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcffae6b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff406ea10 con 0x7fcff412a770 2024-09-13T18:36:42.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcffae6b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff406e230 con 0x7fcff406c590 2024-09-13T18:36:42.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcffae6b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff406d2f0 con 0x7fcff406ceb0 2024-09-13T18:36:42.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fd937fff640 1 -- 172.21.15.177:0/2564711309 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994029822 0 0) 0x7fd94006c4a0 con 0x7fd94006bd60 2024-09-13T18:36:42.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.102+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff406ceb0 0x7fcff412a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fcff9408640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fcff412a770 0x7fcff412abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42688/0 (socket says 172.21.15.177:42688) 2024-09-13T18:36:42.107 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fcff9408640 1 -- 172.21.15.177:0/3359593185 learned_addr learned my addr 172.21.15.177:0/3359593185 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:42.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fd937fff640 1 -- 172.21.15.177:0/2564711309 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c003520 con 0x7fd94006bd60 2024-09-13T18:36:42.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fd945a8d640 1 -- 172.21.15.177:0/2564711309 >> v2:172.21.15.177:3300/0 conn(0x7fd94006b440 msgr2=0x7fd94006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:42.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fd945a8d640 1 --2- 172.21.15.177:0/2564711309 >> v2:172.21.15.177:3300/0 conn(0x7fd94006b440 0x7fd94006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fd945a8d640 1 -- 172.21.15.177:0/2564711309 >> v1:172.21.15.177:6789/0 conn(0x7fd94006bd60 legacy=0x7fd94012d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.108 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fd945a8d640 1 -- 172.21.15.177:0/2564711309 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94006c170 con 0x7fd94012dbf0 2024-09-13T18:36:42.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fcff3fff640 1 -- 172.21.15.177:0/3359593185 >> v2:172.21.15.177:3300/0 conn(0x7fcff406c590 msgr2=0x7fcff406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:42.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fcff3fff640 1 --2- 172.21.15.177:0/3359593185 >> v2:172.21.15.177:3300/0 conn(0x7fcff406c590 0x7fcff406c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fcff3fff640 1 -- 172.21.15.177:0/3359593185 >> v1:172.21.15.177:6789/0 conn(0x7fcff412a770 legacy=0x7fcff412abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7fcff3fff640 1 -- 172.21.15.177:0/3359593185 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff406ede0 con 0x7fcff406ceb0 2024-09-13T18:36:42.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7f5a7d420640 1 Processor -- start 2024-09-13T18:36:42.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.103+0000 7f5a7d420640 1 -- start start 2024-09-13T18:36:42.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a7d420640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 0x7f5a7812a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a7d420640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5a7812abc0 0x7f5a7812d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a7d420640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812daf0 0x7f5a7812df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7fd945a8d640 1 --2- 172.21.15.177:0/2564711309 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94012dbf0 0x7fd94012e040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd938009dc0 tx=0x7fd93802a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f250b0cf55e40f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 0x7f5a7812a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 0x7f5a7812a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60554/0 (socket says 172.21.15.177:60554) 2024-09-13T18:36:42.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 -- 172.21.15.177:0/2864425180 learned_addr learned my addr 172.21.15.177:0/2864425180 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:42.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a777fe640 1 --2- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812daf0 0x7f5a7812df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a767fc640 1 --2- 172.21.15.177:0/2864425180 >> v2:172.21.15.177:3300/0 conn(0x7f5a7812abc0 0x7f5a7812d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a7d420640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7812b040 con 0x7f5a7812abc0 2024-09-13T18:36:42.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7812a470 con 0x7f5a7812daf0 2024-09-13T18:36:42.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78070950 con 0x7f5a7812a090 2024-09-13T18:36:42.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7fd937fff640 1 -- 172.21.15.177:0/2564711309 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938007740 con 0x7fd94012dbf0 2024-09-13T18:36:42.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7fd937fff640 1 -- 172.21.15.177:0/2564711309 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93802d050 con 0x7fd94012dbf0 2024-09-13T18:36:42.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7fd937fff640 1 -- 172.21.15.177:0/2564711309 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938007ac0 con 0x7fd94012dbf0 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 -- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812daf0 msgr2=0x7f5a7812df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 --2- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812daf0 0x7f5a7812df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 -- 172.21.15.177:0/2864425180 >> v2:172.21.15.177:3300/0 conn(0x7f5a7812abc0 msgr2=0x7f5a7812d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 --2- 172.21.15.177:0/2864425180 >> v2:172.21.15.177:3300/0 conn(0x7f5a7812abc0 0x7f5a7812d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.104+0000 7f5a76ffd640 1 -- 172.21.15.177:0/2864425180 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7812e4a0 con 0x7f5a7812a090 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94012dbf0 msgr2=0x7fd94012e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 --2- 172.21.15.177:0/2564711309 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94012dbf0 0x7fd94012e040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd938009dc0 tx=0x7fd93802a560 comp rx=0 tx=0).stop 2024-09-13T18:36:42.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fcff3fff640 1 --2- 172.21.15.177:0/3359593185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff406ceb0 0x7fcff412a230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00097a0 tx=0x7fcfe002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4bede82eb0c2175 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 shutdown_connections 2024-09-13T18:36:42.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 --2- 172.21.15.177:0/2564711309 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94012dbf0 0x7fd94012e040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 --2- 172.21.15.177:0/2564711309 >> v2:172.21.15.177:3300/0 conn(0x7fd94006b440 0x7fd94006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7f5a76ffd640 1 --2- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 0x7f5a7812a680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009870 tx=0x7f5a6802a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb785b2de776467f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 >> 172.21.15.177:0/2564711309 conn(0x7fd94009fe10 msgr2=0x7fd9400a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 shutdown_connections 2024-09-13T18:36:42.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fd9474f0640 1 -- 172.21.15.177:0/2564711309 wait complete. 2024-09-13T18:36:42.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fcff37fe640 1 -- 172.21.15.177:0/3359593185 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe002adb0 con 0x7fcff406ceb0 2024-09-13T18:36:42.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fcff37fe640 1 -- 172.21.15.177:0/3359593185 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe0032440 con 0x7fcff406ceb0 2024-09-13T18:36:42.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7fcff37fe640 1 -- 172.21.15.177:0/3359593185 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00327d0 con 0x7fcff406ceb0 2024-09-13T18:36:42.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7f5a75ffb640 1 -- 172.21.15.177:0/2864425180 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6802ae00 con 0x7f5a7812a090 2024-09-13T18:36:42.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7f5a75ffb640 1 -- 172.21.15.177:0/2864425180 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68032440 con 0x7f5a7812a090 2024-09-13T18:36:42.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.105+0000 7f5a75ffb640 1 -- 172.21.15.177:0/2864425180 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a680327d0 con 0x7f5a7812a090 2024-09-13T18:36:42.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 -- 172.21.15.177:0/3359593185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff406ceb0 msgr2=0x7fcff412a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 --2- 172.21.15.177:0/3359593185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff406ceb0 0x7fcff412a230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00097a0 tx=0x7fcfe002a060 comp rx=0 tx=0).stop 2024-09-13T18:36:42.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 msgr2=0x7f5a7812a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 --2- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 0x7f5a7812a680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009870 tx=0x7f5a6802a0b0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 shutdown_connections 2024-09-13T18:36:42.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fd9474f0640 1 Processor -- start 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 --2- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812daf0 0x7f5a7812df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 -- 172.21.15.177:0/3359593185 shutdown_connections 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 --2- 172.21.15.177:0/3359593185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff406ceb0 0x7fcff412a230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 --2- 172.21.15.177:0/3359593185 >> v2:172.21.15.177:3300/0 conn(0x7fcff406c590 0x7fcff406c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 -- 172.21.15.177:0/3359593185 >> 172.21.15.177:0/3359593185 conn(0x7fcff409ccd0 msgr2=0x7fcff409d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 -- 172.21.15.177:0/3359593185 shutdown_connections 2024-09-13T18:36:42.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fcffae6b640 1 -- 172.21.15.177:0/3359593185 wait complete. 2024-09-13T18:36:42.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7fd9474f0640 1 -- start start 2024-09-13T18:36:42.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 --2- 172.21.15.177:0/2864425180 >> v2:172.21.15.177:3300/0 conn(0x7f5a7812abc0 0x7f5a7812d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 --2- 172.21.15.177:0/2864425180 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812a090 0x7f5a7812a680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 >> 172.21.15.177:0/2864425180 conn(0x7f5a7809cd60 msgr2=0x7f5a7809d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 shutdown_connections 2024-09-13T18:36:42.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.106+0000 7f5a7d420640 1 -- 172.21.15.177:0/2864425180 wait complete. 2024-09-13T18:36:42.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd9474f0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 0x7fd940099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd9474f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd94006bd60 0x7fd940098410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd9474f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 0x7fd940098950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd9474f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400704c0 con 0x7fd94012dbf0 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd9474f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940070780 con 0x7fd94006bd60 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 0x7fd940099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd945a8d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 0x7fd940098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd944a8b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd94006bd60 0x7fd940098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 0x7fd940099ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60562/0 (socket says 172.21.15.177:60562) 2024-09-13T18:36:42.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 -- 172.21.15.177:0/271243589 learned_addr learned my addr 172.21.15.177:0/271243589 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:42.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd945a8d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 0x7fd940098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45938/0 (socket says 172.21.15.177:45938) 2024-09-13T18:36:42.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400975a0 con 0x7fd94006b440 2024-09-13T18:36:42.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 -- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd94006bd60 msgr2=0x7fd940098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.122 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fcffae6b640 1 Processor -- start 2024-09-13T18:36:42.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd94006bd60 0x7fd940098410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 -- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 msgr2=0x7fd940098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 0x7fd940098950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fcffae6b640 1 -- start start 2024-09-13T18:36:42.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd94528c640 1 -- 172.21.15.177:0/271243589 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd938009a70 con 0x7fd94006b440 2024-09-13T18:36:42.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd944a8b640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd94006bd60 0x7fd940098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:42.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.107+0000 7fd945a8d640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 0x7fd940098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:42.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 Processor -- start 2024-09-13T18:36:42.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fd94528c640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 0x7fd940099ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9b0 tx=0x7fd93c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 -- start start 2024-09-13T18:36:42.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c009670 con 0x7fd94006b440 2024-09-13T18:36:42.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c00c9a0 con 0x7fd94006b440 2024-09-13T18:36:42.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd940097e30 con 0x7fd94006b440 2024-09-13T18:36:42.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00cd50 con 0x7fd94006b440 2024-09-13T18:36:42.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcffae6b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 0x7fcff4121bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940096cd0 con 0x7fd94006b440 2024-09-13T18:36:42.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 0x7f5a781257c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcffae6b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 0x7fcff41220f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcffae6b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 0x7fcff4122630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812abc0 0x7f5a78125d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a76ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 0x7f5a781257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812daf0 0x7f5a78126240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78127900 con 0x7f5a7812a090 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78127ae0 con 0x7f5a7812abc0 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a7d420640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7812eed0 con 0x7f5a7812daf0 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a76ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 0x7f5a781257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45956/0 (socket says 172.21.15.177:45956) 2024-09-13T18:36:42.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7f5a76ffd640 1 -- 172.21.15.177:0/3802442095 learned_addr learned my addr 172.21.15.177:0/3802442095 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 0x7fcff41220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 0x7fcff41220f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60568/0 (socket says 172.21.15.177:60568) 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcff8c07640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 0x7fcff4121bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.108+0000 7fcff8c07640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 0x7fcff4121bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45946/0 (socket says 172.21.15.177:45946) 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff9408640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 0x7fcff4122630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff9408640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 0x7fcff4122630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:56972/0 (socket says 172.21.15.177:56972) 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff3fff640 1 -- 172.21.15.177:0/1469961858 learned_addr learned my addr 172.21.15.177:0/1469961858 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:42.128 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41242e0 con 0x7fcff406c590 2024-09-13T18:36:42.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41244e0 con 0x7fcff412a770 2024-09-13T18:36:42.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41232d0 con 0x7fcff406ceb0 2024-09-13T18:36:42.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a767fc640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812abc0 0x7f5a78125d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:42.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a76ffd640 1 -- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812daf0 msgr2=0x7f5a78126240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a76ffd640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812daf0 0x7f5a78126240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.129 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a76ffd640 1 -- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812abc0 msgr2=0x7f5a78125d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff8c07640 1 -- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 msgr2=0x7fcff41220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff8c07640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 0x7fcff41220f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a76ffd640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812abc0 0x7f5a78125d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.130 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a76ffd640 1 -- 172.21.15.177:0/3802442095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680094d0 con 0x7f5a7812a090 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff8c07640 1 -- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 msgr2=0x7fcff4122630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff8c07640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 0x7fcff4122630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff8c07640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe0009400 con 0x7fcff406c590 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff9408640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 0x7fcff4122630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff3fff640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 0x7fcff41220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7f5a76ffd640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 0x7f5a781257c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a68000c00 tx=0x7f5a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.131 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9400008d0 con 0x7fd94006b440 2024-09-13T18:36:42.132 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.109+0000 7fcff8c07640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 0x7fcff4121bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800d5f0 tx=0x7fcfe800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:42.133 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7f5a767fc640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812abc0 0x7f5a78125d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:42.133 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6803b600 con 0x7f5a7812a090 2024-09-13T18:36:42.133 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6803b760 con 0x7f5a7812a090 2024-09-13T18:36:42.133 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80125f0 con 0x7fcff406c590 2024-09-13T18:36:42.133 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fd93c013470 con 0x7fd94006b440 2024-09-13T18:36:42.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6803baf0 con 0x7f5a7812a090 2024-09-13T18:36:42.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a78127d80 con 0x7f5a7812a090 2024-09-13T18:36:42.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a78126780 con 0x7f5a7812a090 2024-09-13T18:36:42.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8012750 con 0x7fcff406c590 2024-09-13T18:36:42.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8012b00 con 0x7fcff406c590 2024-09-13T18:36:42.135 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff4124a70 con 0x7fcff406c590 2024-09-13T18:36:42.135 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.110+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff41c6ac0 con 0x7fcff406c590 2024-09-13T18:36:42.135 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.111+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fd93c05d2c0 con 0x7fd94006b440 2024-09-13T18:36:42.135 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.111+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5a68039040 con 0x7f5a7812a090 2024-09-13T18:36:42.135 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.112+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fcfe8012c60 con 0x7fcff406c590 2024-09-13T18:36:42.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.112+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f5a6807cb80 con 0x7f5a7812a090 2024-09-13T18:36:42.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.112+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff4124c60 con 0x7fcff406c590 2024-09-13T18:36:42.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.112+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7812d5b0 con 0x7f5a7812a090 2024-09-13T18:36:42.136 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.112+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fcfe805ce50 con 0x7fcff406c590 2024-09-13T18:36:42.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.115+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fd93c03afb0 con 0x7fd94006b440 2024-09-13T18:36:42.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.116+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fcfe803abe0 con 0x7fcff406c590 2024-09-13T18:36:42.137 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.123+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f5a68037070 con 0x7f5a7812a090 2024-09-13T18:36:42.229 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.227+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4b8406b820 con 0x7f4b8412da20 2024-09-13T18:36:42.229 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.227+0000 7f4b6bfff640 1 -- 172.21.15.177:0/1033072097 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b7c017030 con 0x7f4b8412da20 2024-09-13T18:36:42.230 INFO:teuthology.orchestra.run.smithi177.stdout:120259084290 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 >> v1:172.21.15.177:6816/516212916 conn(0x7f4b70047560 legacy=0x7f4b70049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 msgr2=0x7f4b8412f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 0x7f4b8412f030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c001560 tx=0x7f4b7c00cbe0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 shutdown_connections 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8412da20 0x7f4b8412f030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f4b8406bd60 0x7f4b8412eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 --2- 172.21.15.177:0/1033072097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b8406b440 0x7f4b8412e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.229+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 >> 172.21.15.177:0/1033072097 conn(0x7f4b8409fe10 msgr2=0x7f4b84127290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.230+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 shutdown_connections 2024-09-13T18:36:42.232 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.230+0000 7f4b89ee9640 1 -- 172.21.15.177:0/1033072097 wait complete. 2024-09-13T18:36:42.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084291 got 120259084290 for osd.2 2024-09-13T18:36:42.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.258+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd940135820 con 0x7fd94006b440 2024-09-13T18:36:42.260 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.258+0000 7fd935ffb640 1 -- 172.21.15.177:0/271243589 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd93c021070 con 0x7fd94006b440 2024-09-13T18:36:42.260 INFO:teuthology.orchestra.run.smithi177.stdout:124554051586 2024-09-13T18:36:42.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 >> v1:172.21.15.177:6816/516212916 conn(0x7fd924047560 legacy=0x7fd924049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 msgr2=0x7fd940099ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 0x7fd940099ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9b0 tx=0x7fd93c00ee70 comp rx=0 tx=0).stop 2024-09-13T18:36:42.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 shutdown_connections 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94012dbf0 0x7fd940098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd94006bd60 0x7fd940098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 --2- 172.21.15.177:0/271243589 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fd94006b440 0x7fd940099ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 >> 172.21.15.177:0/271243589 conn(0x7fd94009fe10 msgr2=0x7fd94012c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 shutdown_connections 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.260+0000 7fd9474f0640 1 -- 172.21.15.177:0/271243589 wait complete. 2024-09-13T18:36:42.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.261+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcff4124060 con 0x7fcff406c590 2024-09-13T18:36:42.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.262+0000 7fcff17fa640 1 -- 172.21.15.177:0/1469961858 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcff4124060 con 0x7fcff406c590 2024-09-13T18:36:42.264 INFO:teuthology.orchestra.run.smithi177.stdout:111669149698 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.263+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 >> v1:172.21.15.177:6816/516212916 conn(0x7fcfd8047590 legacy=0x7fcfd8049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.263+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 msgr2=0x7fcff4121bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.263+0000 7fcffae6b640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 0x7fcff4121bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800d5f0 tx=0x7fcfe800dab0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 shutdown_connections 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcff412a770 0x7fcff4122630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fcff406ceb0 0x7fcff41220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 --2- 172.21.15.177:0/1469961858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcff406c590 0x7fcff4121bb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 >> 172.21.15.177:0/1469961858 conn(0x7fcff409ccd0 msgr2=0x7fcff4133210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 shutdown_connections 2024-09-13T18:36:42.266 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.264+0000 7fcffae6b640 1 -- 172.21.15.177:0/1469961858 wait complete. 2024-09-13T18:36:42.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051587 got 124554051586 for osd.3 2024-09-13T18:36:42.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 111669149699 got 111669149698 for osd.0 2024-09-13T18:36:42.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.275+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5a781270a0 con 0x7f5a7812a090 2024-09-13T18:36:42.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.275+0000 7f5a5f7fe640 1 -- 172.21.15.177:0/3802442095 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a68047620 con 0x7f5a7812a090 2024-09-13T18:36:42.278 INFO:teuthology.orchestra.run.smithi177.stdout:115964116994 2024-09-13T18:36:42.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 >> v1:172.21.15.177:6816/516212916 conn(0x7f5a64047590 legacy=0x7f5a64049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 msgr2=0x7f5a781257c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:42.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 0x7f5a781257c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a68000c00 tx=0x7f5a680046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 shutdown_connections 2024-09-13T18:36:42.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5a7812daf0 0x7f5a78126240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a7812abc0 0x7f5a78125d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 --2- 172.21.15.177:0/3802442095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a7812a090 0x7f5a781257c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:42.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 >> 172.21.15.177:0/3802442095 conn(0x7f5a7809cd60 msgr2=0x7f5a78095540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:42.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 shutdown_connections 2024-09-13T18:36:42.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:42.277+0000 7f5a7d420640 1 -- 172.21.15.177:0/3802442095 wait complete. 2024-09-13T18:36:42.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.1 2024-09-13T18:36:43.247 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:36:43.275 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:36:43.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:36:43.291 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:36:43.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.354+0000 7f2aab9aa640 1 Processor -- start 2024-09-13T18:36:43.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.354+0000 7f2aab9aa640 1 -- start start 2024-09-13T18:36:43.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aab9aa640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aab9aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa406bd60 0x7f2aa412d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa9746640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa9746640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa406b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60588/0 (socket says 172.21.15.177:60588) 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa9746640 1 -- 172.21.15.177:0/1946840778 learned_addr learned my addr 172.21.15.177:0/1946840778 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aab9aa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa406c4a0 con 0x7f2aa412da20 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa9f47640 1 --1- 172.21.15.177:0/1946840778 >> v1:172.21.15.177:6789/0 conn(0x7f2aa412da20 0x7f2aa412dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42704/0 (socket says 172.21.15.177:42704) 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa406d900 con 0x7f2aa406bd60 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa406db60 con 0x7f2aa406b440 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa8f45640 1 --2- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa406bd60 0x7f2aa412d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2a9bfff640 1 -- 172.21.15.177:0/1946840778 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742914952 0 0) 0x7f2aa406c4a0 con 0x7f2aa412da20 2024-09-13T18:36:43.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2a9bfff640 1 -- 172.21.15.177:0/1946840778 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a90003520 con 0x7f2aa412da20 2024-09-13T18:36:43.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa8f45640 1 -- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 msgr2=0x7f2aa406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa8f45640 1 --2- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa406b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa8f45640 1 -- 172.21.15.177:0/1946840778 >> v1:172.21.15.177:6789/0 conn(0x7f2aa412da20 legacy=0x7f2aa412dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.355+0000 7f2aa8f45640 1 -- 172.21.15.177:0/1946840778 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa406c140 con 0x7f2aa406bd60 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.357+0000 7f2aa8f45640 1 --2- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa406bd60 0x7f2aa412d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097d0 tx=0x7f2a9402a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9d8bafa406325a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.357+0000 7f2a9bfff640 1 -- 172.21.15.177:0/1946840778 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9402ad90 con 0x7f2aa406bd60 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.357+0000 7f2a9bfff640 1 -- 172.21.15.177:0/1946840778 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94032440 con 0x7f2aa406bd60 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.357+0000 7f2a9bfff640 1 -- 172.21.15.177:0/1946840778 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940327d0 con 0x7f2aa406bd60 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.357+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa406bd60 msgr2=0x7f2aa412d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.357+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa406bd60 0x7f2aa412d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097d0 tx=0x7f2a9402a040 comp rx=0 tx=0).stop 2024-09-13T18:36:43.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.358+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 shutdown_connections 2024-09-13T18:36:43.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.358+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa406bd60 0x7f2aa412d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.358+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/1946840778 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa406b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.358+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 >> 172.21.15.177:0/1946840778 conn(0x7f2aa409fe10 msgr2=0x7f2aa40a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.358+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 shutdown_connections 2024-09-13T18:36:43.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.358+0000 7f2aab9aa640 1 -- 172.21.15.177:0/1946840778 wait complete. 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 Processor -- start 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 -- start start 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa4097df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa406bd60 0x7f2aa4098330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 0x7f2aa4098870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa4099fa0 con 0x7f2aa406bd60 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa408bcd0 con 0x7f2aa412da20 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aab9aa640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa409a270 con 0x7f2aa406b440 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9f47640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 0x7f2aa4098870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9746640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa4097df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9f47640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 0x7f2aa4098870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57014/0 (socket says 172.21.15.177:57014) 2024-09-13T18:36:43.363 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9f47640 1 -- 172.21.15.177:0/2172095130 learned_addr learned my addr 172.21.15.177:0/2172095130 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9746640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa4097df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60592/0 (socket says 172.21.15.177:60592) 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa8f45640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa406bd60 0x7f2aa4098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9f47640 1 -- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 msgr2=0x7f2aa4097df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9f47640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa4097df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.359+0000 7f2aa9f47640 1 -- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa406bd60 msgr2=0x7f2aa4098330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2aa9f47640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa406bd60 0x7f2aa4098330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2aa9f47640 1 -- 172.21.15.177:0/2172095130 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009400 con 0x7f2aa412da20 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2aa8f45640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa406bd60 0x7f2aa4098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2aa9f47640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 0x7f2aa4098870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c7c0 tx=0x7f2a9c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00a640 con 0x7f2aa412da20 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c00a7a0 con 0x7f2aa412da20 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa408b1b0 con 0x7f2aa412da20 2024-09-13T18:36:43.364 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00ab30 con 0x7f2aa412da20 2024-09-13T18:36:43.365 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.360+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa4098e60 con 0x7f2aa412da20 2024-09-13T18:36:43.365 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.362+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa406b820 con 0x7f2aa412da20 2024-09-13T18:36:43.365 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.362+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f2a9c00ac90 con 0x7f2aa412da20 2024-09-13T18:36:43.365 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.362+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f2a9c05bff0 con 0x7f2aa412da20 2024-09-13T18:36:43.371 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.369+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f2a9c039c00 con 0x7f2aa412da20 2024-09-13T18:36:43.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 Processor -- start 2024-09-13T18:36:43.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 Processor -- start 2024-09-13T18:36:43.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 -- start start 2024-09-13T18:36:43.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 -- start start 2024-09-13T18:36:43.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fee64129eb0 0x7fee6412a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 0x7f013c12f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6406d1e0 con 0x7fee64129eb0 2024-09-13T18:36:43.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c12f9d0 0x7f013c131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee64066f90 con 0x7fee6412d8f0 2024-09-13T18:36:43.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013c12a2f0 con 0x7f013c12a4d0 2024-09-13T18:36:43.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f0141b8f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 0x7f013c12f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6ad9d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6406da20 con 0x7fee6412a7d0 2024-09-13T18:36:43.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6959a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c097e00 con 0x7f013c12f9d0 2024-09-13T18:36:43.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f0141b8f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 0x7f013c12f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60608/0 (socket says 172.21.15.177:60608) 2024-09-13T18:36:43.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f0141b8f640 1 -- 172.21.15.177:0/2109607833 learned_addr learned my addr 172.21.15.177:0/2109607833 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee69d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fee64129eb0 0x7fee6412a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7fee6a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.396+0000 7f01445f4640 1 -- 172.21.15.177:0/2109607833 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f013c12afc0 con 0x7f013c06e090 2024-09-13T18:36:43.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee69d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fee64129eb0 0x7fee6412a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45972/0 (socket says 172.21.15.177:45972) 2024-09-13T18:36:43.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee69d9b640 1 -- 172.21.15.177:0/4156183753 learned_addr learned my addr 172.21.15.177:0/4156183753 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee6a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57030/0 (socket says 172.21.15.177:57030) 2024-09-13T18:36:43.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7f0142b91640 1 --2- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c12f9d0 0x7f013c131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee6a59c640 1 -- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 msgr2=0x7fee6412d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee6a59c640 1 --2- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7f0141b8f640 1 -- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c12f9d0 msgr2=0x7f013c131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee6a59c640 1 -- 172.21.15.177:0/4156183753 >> v2:172.21.15.177:3300/0 conn(0x7fee64129eb0 msgr2=0x7fee6412a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee6a59c640 1 --2- 172.21.15.177:0/4156183753 >> v2:172.21.15.177:3300/0 conn(0x7fee64129eb0 0x7fee6412a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7f0141b8f640 1 --2- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c12f9d0 0x7f013c131dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7f0141b8f640 1 -- 172.21.15.177:0/2109607833 >> v1:172.21.15.177:6789/0 conn(0x7f013c12a4d0 legacy=0x7f013c12a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7f0141b8f640 1 -- 172.21.15.177:0/2109607833 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c06e870 con 0x7f013c06e090 2024-09-13T18:36:43.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.397+0000 7fee6a59c640 1 -- 172.21.15.177:0/4156183753 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6412e2a0 con 0x7fee6412d8f0 2024-09-13T18:36:43.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7f0141b8f640 1 --2- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 0x7f013c12f490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f012c0098a0 tx=0x7f012c02c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee010363513122d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7f014138e640 1 -- 172.21.15.177:0/2109607833 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c034440 con 0x7f013c06e090 2024-09-13T18:36:43.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7fee6a59c640 1 --2- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412dd60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009dc0 tx=0x7fee5c02a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9657c32cde40d7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7f014138e640 1 -- 172.21.15.177:0/2109607833 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c0345a0 con 0x7f013c06e090 2024-09-13T18:36:43.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7f9360d25640 1 Processor -- start 2024-09-13T18:36:43.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7f014138e640 1 -- 172.21.15.177:0/2109607833 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c034930 con 0x7f013c06e090 2024-09-13T18:36:43.407 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.398+0000 7f9360d25640 1 -- start start 2024-09-13T18:36:43.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7fee68d99640 1 -- 172.21.15.177:0/4156183753 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007740 con 0x7fee6412d8f0 2024-09-13T18:36:43.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7fee68d99640 1 -- 172.21.15.177:0/4156183753 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c02d050 con 0x7fee6412d8f0 2024-09-13T18:36:43.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7fee68d99640 1 -- 172.21.15.177:0/4156183753 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007ac0 con 0x7fee6412d8f0 2024-09-13T18:36:43.408 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 -- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 msgr2=0x7f013c12f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 --2- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 0x7f013c12f490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f012c0098a0 tx=0x7f012c02c220 comp rx=0 tx=0).stop 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 -- 172.21.15.177:0/2109607833 shutdown_connections 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 --2- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c12f9d0 0x7f013c131dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 --2- 172.21.15.177:0/2109607833 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c06e090 0x7f013c12f490 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 -- 172.21.15.177:0/2109607833 >> 172.21.15.177:0/2109607833 conn(0x7f013c098ba0 msgr2=0x7f013c096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f9360d25640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c126490 0x7f935c1269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.409 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f9360d25640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f935c0705b0 0x7f935c126ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f9360d25640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c1299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f9360d25640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c06fde0 con 0x7f935c0705b0 2024-09-13T18:36:43.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f9360d25640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c127420 con 0x7f935c126490 2024-09-13T18:36:43.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f9360d25640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f935c070d20 con 0x7f935c069ed0 2024-09-13T18:36:43.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 -- 172.21.15.177:0/2109607833 shutdown_connections 2024-09-13T18:36:43.410 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f935a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c126490 0x7f935c1269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.399+0000 7f01445f4640 1 -- 172.21.15.177:0/2109607833 wait complete. 2024-09-13T18:36:43.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f935a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c126490 0x7f935c1269a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57058/0 (socket says 172.21.15.177:57058) 2024-09-13T18:36:43.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f935a59c640 1 -- 172.21.15.177:0/2071484685 learned_addr learned my addr 172.21.15.177:0/2071484685 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f935ad9d640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c1299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f9359d9b640 1 --2- 172.21.15.177:0/2071484685 >> v2:172.21.15.177:3300/0 conn(0x7f935c0705b0 0x7f935c126ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.411 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f9359d9b640 1 -- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 msgr2=0x7f935c1299b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f9359d9b640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c1299b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f9359d9b640 1 -- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c126490 msgr2=0x7f935c1269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 -- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 msgr2=0x7fee6412dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f9359d9b640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c126490 0x7f935c1269a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.412 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412dd60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009dc0 tx=0x7fee5c02a4d0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f9359d9b640 1 -- 172.21.15.177:0/2071484685 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c06a360 con 0x7f935c0705b0 2024-09-13T18:36:43.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7f935ad9d640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c1299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.413 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 -- 172.21.15.177:0/4156183753 shutdown_connections 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412dd60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/4156183753 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/4156183753 >> v2:172.21.15.177:3300/0 conn(0x7fee64129eb0 0x7fee6412a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 -- 172.21.15.177:0/4156183753 >> 172.21.15.177:0/4156183753 conn(0x7fee6409a730 msgr2=0x7fee6409ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 -- 172.21.15.177:0/4156183753 shutdown_connections 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.400+0000 7fee6ad9d640 1 -- 172.21.15.177:0/4156183753 wait complete. 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f01445f4640 1 Processor -- start 2024-09-13T18:36:43.414 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f01445f4640 1 -- start start 2024-09-13T18:36:43.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f9359d9b640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c0705b0 0x7f935c126ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93500097a0 tx=0x7f9350029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2e187025d71f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f01445f4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c06e090 0x7f013c125c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7fee6ad9d640 1 Processor -- start 2024-09-13T18:36:43.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7fee6ad9d640 1 -- start start 2024-09-13T18:36:43.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f01445f4640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c12a4d0 0x7f013c1261b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f0142390640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c06e090 0x7f013c125c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f935959a640 1 -- 172.21.15.177:0/2071484685 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935002ad20 con 0x7f935c0705b0 2024-09-13T18:36:43.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f0142390640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c06e090 0x7f013c125c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57064/0 (socket says 172.21.15.177:57064) 2024-09-13T18:36:43.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f935959a640 1 -- 172.21.15.177:0/2071484685 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935002ae80 con 0x7f935c0705b0 2024-09-13T18:36:43.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.401+0000 7f935959a640 1 -- 172.21.15.177:0/2071484685 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350032660 con 0x7f935c0705b0 2024-09-13T18:36:43.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0142390640 1 -- 172.21.15.177:0/3872079735 learned_addr learned my addr 172.21.15.177:0/3872079735 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0141b8f640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c12a4d0 0x7f013c1261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f01445f4640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013c12f9d0 0x7f013c1266f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c123dc0 con 0x7f013c12f9d0 2024-09-13T18:36:43.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c1250e0 con 0x7f013c06e090 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0712f0 con 0x7f013c12a4d0 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0142b91640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013c12f9d0 0x7f013c1266f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0141b8f640 1 -- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c06e090 msgr2=0x7f013c125c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0141b8f640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c06e090 0x7f013c125c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0141b8f640 1 -- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013c12f9d0 msgr2=0x7f013c1266f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f9360d25640 1 -- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c0705b0 msgr2=0x7f935c126ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f9360d25640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c0705b0 0x7f935c126ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93500097a0 tx=0x7f9350029fd0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0141b8f640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013c12f9d0 0x7f013c1266f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7f0141b8f640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0094d0 con 0x7f013c12a4d0 2024-09-13T18:36:43.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7fee6ad9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 0x7fee6412e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7fee6ad9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7fee69d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 0x7fee6412e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7fee69d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 0x7fee6412e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45998/0 (socket says 172.21.15.177:45998) 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7fee6959a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.402+0000 7fee6959a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60658/0 (socket says 172.21.15.177:60658) 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6ad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6959a640 1 -- 172.21.15.177:0/448508170 learned_addr learned my addr 172.21.15.177:0/448508170 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6412f6f0 con 0x7fee64129eb0 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640712f0 con 0x7fee6412d8f0 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee641303d0 con 0x7fee6412a7d0 2024-09-13T18:36:43.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f0141b8f640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c12a4d0 0x7f013c1261b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f013c12b150 tx=0x7f012c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 -- 172.21.15.177:0/2071484685 shutdown_connections 2024-09-13T18:36:43.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c034440 con 0x7f013c12a4d0 2024-09-13T18:36:43.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c1299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c0705b0 0x7f935c126ee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 --2- 172.21.15.177:0/2071484685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c126490 0x7f935c1269a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.423 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 -- 172.21.15.177:0/2071484685 >> 172.21.15.177:0/2071484685 conn(0x7f935c09b680 msgr2=0x7f935c09ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.423 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6a59c640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.423 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c032ba0 con 0x7f013c12a4d0 2024-09-13T18:36:43.423 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c03e400 con 0x7f013c12a4d0 2024-09-13T18:36:43.423 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013c123380 con 0x7f013c12a4d0 2024-09-13T18:36:43.424 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6959a640 1 -- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 msgr2=0x7fee6412eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.424 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6959a640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.424 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7fee6959a640 1 -- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 msgr2=0x7fee6412e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.424 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c127720 con 0x7f013c12a4d0 2024-09-13T18:36:43.425 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 -- 172.21.15.177:0/2071484685 shutdown_connections 2024-09-13T18:36:43.425 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.403+0000 7f9360d25640 1 -- 172.21.15.177:0/2071484685 wait complete. 2024-09-13T18:36:43.425 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee6959a640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 0x7fee6412e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.425 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee6959a640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009a70 con 0x7fee6412a7d0 2024-09-13T18:36:43.426 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee69d9b640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 0x7fee6412e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.426 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee6959a640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412e970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9b0 tx=0x7fee5400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.426 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f012c03e560 con 0x7f013c12a4d0 2024-09-13T18:36:43.426 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54009670 con 0x7fee6412a7d0 2024-09-13T18:36:43.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5400c9a0 con 0x7fee6412a7d0 2024-09-13T18:36:43.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400cd50 con 0x7fee6412a7d0 2024-09-13T18:36:43.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6406c980 con 0x7fee6412a7d0 2024-09-13T18:36:43.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee64130b40 con 0x7fee6412a7d0 2024-09-13T18:36:43.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.404+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f012c07d410 con 0x7f013c12a4d0 2024-09-13T18:36:43.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c123510 con 0x7f013c12a4d0 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 Processor -- start 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 -- start start 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c0705b0 0x7f935c115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c126490 0x7f935c115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c118810 con 0x7f935c126490 2024-09-13T18:36:43.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1183f0 con 0x7f935c0705b0 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.405+0000 7f9360d25640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1177e0 con 0x7f935c069ed0 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935a59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935a59c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c115550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60666/0 (socket says 172.21.15.177:60666) 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935a59c640 1 -- 172.21.15.177:0/2099292006 learned_addr learned my addr 172.21.15.177:0/2099292006 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fee54013470 con 0x7fee6412a7d0 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c126490 0x7f935c115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f9359d9b640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c0705b0 0x7f935c115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6412d3b0 con 0x7fee6412a7d0 2024-09-13T18:36:43.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 -- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 msgr2=0x7f935c115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 -- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c0705b0 msgr2=0x7f935c115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c0705b0 0x7f935c115a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 -- 172.21.15.177:0/2099292006 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935400bc80 con 0x7f935c126490 2024-09-13T18:36:43.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f9359d9b640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c0705b0 0x7f935c115a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:43.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fee5405cae0 con 0x7fee6412a7d0 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935a59c640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.406+0000 7f935ad9d640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c126490 0x7f935c115fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93540092f0 tx=0x7f935400d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.407+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400b2d0 con 0x7f935c126490 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.407+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935400de80 con 0x7f935c126490 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.407+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9350009400 con 0x7f935c126490 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.408+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c116ee0 con 0x7f935c126490 2024-09-13T18:36:43.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.408+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354003bd0 con 0x7f935c126490 2024-09-13T18:36:43.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.408+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9354009540 con 0x7f935c126490 2024-09-13T18:36:43.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.408+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c128c40 con 0x7f935c126490 2024-09-13T18:36:43.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.408+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f9354061db0 con 0x7f935c126490 2024-09-13T18:36:43.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.411+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f013c123510 con 0x7f013c12a4d0 2024-09-13T18:36:43.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.417+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f9354062060 con 0x7f935c126490 2024-09-13T18:36:43.433 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.418+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fee5403a670 con 0x7fee6412a7d0 2024-09-13T18:36:43.520 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.517+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2aa4098e60 con 0x7f2aa412da20 2024-09-13T18:36:43.520 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.518+0000 7f2a99ffb640 1 -- 172.21.15.177:0/2172095130 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2aa4098e60 con 0x7f2aa412da20 2024-09-13T18:36:43.520 INFO:teuthology.orchestra.run.smithi177.stdout:120259084291 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 >> v1:172.21.15.177:6816/516212916 conn(0x7f2a88047560 legacy=0x7f2a88049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 msgr2=0x7f2aa4098870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 0x7f2aa4098870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c7c0 tx=0x7f2a9c00cc80 comp rx=0 tx=0).stop 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 shutdown_connections 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa412da20 0x7f2aa4098870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa406bd60 0x7f2aa4098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 --2- 172.21.15.177:0/2172095130 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2aa406b440 0x7f2aa4097df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.522 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 >> 172.21.15.177:0/2172095130 conn(0x7f2aa409fe10 msgr2=0x7f2aa409b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.523 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 shutdown_connections 2024-09-13T18:36:43.523 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.520+0000 7f2aab9aa640 1 -- 172.21.15.177:0/2172095130 wait complete. 2024-09-13T18:36:43.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084291 got 120259084291 for osd.2 2024-09-13T18:36:43.537 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:43.568 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.566+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee64073cc0 con 0x7fee6412a7d0 2024-09-13T18:36:43.569 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.566+0000 7fee5a7fc640 1 -- 172.21.15.177:0/448508170 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee5403dc90 con 0x7fee6412a7d0 2024-09-13T18:36:43.569 INFO:teuthology.orchestra.run.smithi177.stdout:124554051587 2024-09-13T18:36:43.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 >> v1:172.21.15.177:6816/516212916 conn(0x7fee48047560 legacy=0x7fee48049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 msgr2=0x7fee6412e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412e970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9b0 tx=0x7fee5400ee70 comp rx=0 tx=0).stop 2024-09-13T18:36:43.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 shutdown_connections 2024-09-13T18:36:43.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee6412d8f0 0x7fee6412eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.571 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fee6412a7d0 0x7fee6412e970 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.572 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 --2- 172.21.15.177:0/448508170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee64129eb0 0x7fee6412e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.572 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.568+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 >> 172.21.15.177:0/448508170 conn(0x7fee6409a730 msgr2=0x7fee6412c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.572 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.569+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 shutdown_connections 2024-09-13T18:36:43.572 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.569+0000 7fee6ad9d640 1 -- 172.21.15.177:0/448508170 wait complete. 2024-09-13T18:36:43.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.572+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f013c123510 con 0x7f013c12a4d0 2024-09-13T18:36:43.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.572+0000 7f0132ffd640 1 -- 172.21.15.177:0/3872079735 <== mon.2 v2:172.21.15.177:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f013c123510 con 0x7f013c12a4d0 2024-09-13T18:36:43.574 INFO:teuthology.orchestra.run.smithi177.stdout:115964116995 2024-09-13T18:36:43.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 >> v1:172.21.15.177:6816/516212916 conn(0x7f01200475b0 legacy=0x7f0120049a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c12a4d0 msgr2=0x7f013c1261b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c12a4d0 0x7f013c1261b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f013c12b150 tx=0x7f012c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 shutdown_connections 2024-09-13T18:36:43.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013c12f9d0 0x7f013c1266f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f013c12a4d0 0x7f013c1261b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 --2- 172.21.15.177:0/3872079735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f013c06e090 0x7f013c125c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 >> 172.21.15.177:0/3872079735 conn(0x7f013c098ba0 msgr2=0x7f013c09a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 shutdown_connections 2024-09-13T18:36:43.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.574+0000 7f01445f4640 1 -- 172.21.15.177:0/3872079735 wait complete. 2024-09-13T18:36:43.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.576+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f935c12a680 con 0x7f935c126490 2024-09-13T18:36:43.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.577+0000 7f934affd640 1 -- 172.21.15.177:0/2099292006 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f935403a380 con 0x7f935c126490 2024-09-13T18:36:43.579 INFO:teuthology.orchestra.run.smithi177.stdout:111669149700 2024-09-13T18:36:43.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 >> v1:172.21.15.177:6816/516212916 conn(0x7f933c047560 legacy=0x7f933c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c126490 msgr2=0x7f935c115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c126490 0x7f935c115fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93540092f0 tx=0x7f935400d480 comp rx=0 tx=0).stop 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 shutdown_connections 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c126490 0x7f935c115fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f935c0705b0 0x7f935c115a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 --2- 172.21.15.177:0/2099292006 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f935c069ed0 0x7f935c115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 >> 172.21.15.177:0/2099292006 conn(0x7f935c09b680 msgr2=0x7f935c125020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 shutdown_connections 2024-09-13T18:36:43.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.579+0000 7f9360d25640 1 -- 172.21.15.177:0/2099292006 wait complete. 2024-09-13T18:36:43.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051587 got 124554051587 for osd.3 2024-09-13T18:36:43.583 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:43.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116995 for osd.1 2024-09-13T18:36:43.588 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:43.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 111669149699 got 111669149700 for osd.0 2024-09-13T18:36:43.593 DEBUG:teuthology.parallel:result is None 2024-09-13T18:36:43.593 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:36:43.593 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 Processor -- start 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 -- start start 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88d812d4d0 0x7f88d812f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d812fe00 0x7f88d81321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d806c460 con 0x7f88d806c640 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d8097e00 con 0x7f88d812d4d0 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.729+0000 7f88e04d7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88d806d770 con 0x7f88d812fe00 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dda72640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88d812d4d0 0x7f88d812f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dea74640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d812fe00 0x7f88d81321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88de273640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f88d806c640 0x7f88d806ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42716/0 (socket says 172.21.15.177:42716) 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88de273640 1 -- 172.21.15.177:0/945153994 learned_addr learned my addr 172.21.15.177:0/945153994 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dda72640 1 -- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d812fe00 msgr2=0x7f88d81321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dda72640 1 --2- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d812fe00 0x7f88d81321f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dda72640 1 -- 172.21.15.177:0/945153994 >> v1:172.21.15.177:6789/0 conn(0x7f88d806c640 legacy=0x7f88d806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dda72640 1 -- 172.21.15.177:0/945153994 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d806e0b0 con 0x7f88d812d4d0 2024-09-13T18:36:43.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.730+0000 7f88dd271640 1 -- 172.21.15.177:0/945153994 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002476131 0 0) 0x7f88d806c460 con 0x7f88d806c640 2024-09-13T18:36:43.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88dda72640 1 --2- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d812f8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f88c80097d0 tx=0x7f88c802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a4515c05223d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88dd271640 1 -- 172.21.15.177:0/945153994 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c802adb0 con 0x7f88d812d4d0 2024-09-13T18:36:43.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88dd271640 1 -- 172.21.15.177:0/945153994 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c8032440 con 0x7f88d812d4d0 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88dd271640 1 -- 172.21.15.177:0/945153994 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c80327d0 con 0x7f88d812d4d0 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 -- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 msgr2=0x7f88d812f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 --2- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d812f8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f88c80097d0 tx=0x7f88c802a0a0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 -- 172.21.15.177:0/945153994 shutdown_connections 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 --2- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d812f8c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 --2- 172.21.15.177:0/945153994 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d812fe00 0x7f88d81321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 -- 172.21.15.177:0/945153994 >> 172.21.15.177:0/945153994 conn(0x7f88d8098ba0 msgr2=0x7f88d8096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 -- 172.21.15.177:0/945153994 shutdown_connections 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.731+0000 7f88e04d7640 1 -- 172.21.15.177:0/945153994 wait complete. 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 Processor -- start 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 -- start start 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d806c640 0x7f88d80953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.737 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d8095900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d812fe00 0x7f88d8127fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81243d0 con 0x7f88d812d4d0 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.732+0000 7f88e04d7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d8123d00 con 0x7f88d812fe00 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88e04d7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81248e0 con 0x7f88d806c640 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d8095900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88de273640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d806c640 0x7f88d80953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d8095900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46036/0 (socket says 172.21.15.177:46036) 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 -- 172.21.15.177:0/2968393411 learned_addr learned my addr 172.21.15.177:0/2968393411 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dea74640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d812fe00 0x7f88d8127fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 -- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d806c640 msgr2=0x7f88d80953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d806c640 0x7f88d80953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 -- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d812fe00 msgr2=0x7f88d8127fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d812fe00 0x7f88d8127fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 -- 172.21.15.177:0/2968393411 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8009400 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88de273640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d806c640 0x7f88d80953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dea74640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d812fe00 0x7f88d8127fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88dda72640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d8095900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88c802a5c0 tx=0x7f88c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c803d650 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c803d7d0 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d8124b70 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c803db80 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.733+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d8095e40 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.734+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f88c8039040 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.734+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d806ca20 con 0x7f88d812d4d0 2024-09-13T18:36:43.739 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.735+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f88c807d0c0 con 0x7f88d812d4d0 2024-09-13T18:36:43.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.738+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f88c8007cc0 con 0x7f88d812d4d0 2024-09-13T18:36:43.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.870+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88d806b1d0 con 0x7f88bc0475c0 2024-09-13T18:36:43.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.871+0000 7f88ceffd640 1 -- 172.21.15.177:0/2968393411 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34788 (unknown 2965378022 0 3289417995) 0x7f88d4001660 con 0x7f88bc0475c0 2024-09-13T18:36:43.873 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:43.874 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 >> v1:172.21.15.177:6816/516212916 conn(0x7f88bc0475c0 legacy=0x7f88bc049a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 msgr2=0x7f88d8095900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d8095900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88c802a5c0 tx=0x7f88c80046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 shutdown_connections 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d812fe00 0x7f88d8127fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d812d4d0 0x7f88d8095900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 --2- 172.21.15.177:0/2968393411 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f88d806c640 0x7f88d80953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 >> 172.21.15.177:0/2968393411 conn(0x7f88d8098ba0 msgr2=0x7f88d806ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 shutdown_connections 2024-09-13T18:36:43.876 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.873+0000 7f88e04d7640 1 -- 172.21.15.177:0/2968393411 wait complete. 2024-09-13T18:36:43.885 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-13T18:36:42.593773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112316,"kb_used_data":2244,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375044,"statfs":{"total":386547056640,"available":386432045056,"internally_reserved":0,"allocated":2297856,"data_stored":1566874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.941701"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889716+0000","last_change":"2024-09-13T18:36:36.889220+0000","last_active":"2024-09-13T18:36:36.889716+0000","last_peered":"2024-09-13T18:36:36.889716+0000","last_clean":"2024-09-13T18:36:36.889716+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:36:36.889716+0000","last_undegraded":"2024-09-13T18:36:36.889716+0000","last_fullsized":"2024-09-13T18:36:36.889716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:19.625333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664807+0000","last_change":"2024-09-13T18:36:34.636621+0000","last_active":"2024-09-13T18:36:38.664807+0000","last_peered":"2024-09-13T18:36:38.664807+0000","last_clean":"2024-09-13T18:36:38.664807+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:38.664807+0000","last_undegraded":"2024-09-13T18:36:38.664807+0000","last_fullsized":"2024-09-13T18:36:38.664807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:09.196966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889523+0000","last_change":"2024-09-13T18:36:36.888348+0000","last_active":"2024-09-13T18:36:36.889523+0000","last_peered":"2024-09-13T18:36:36.889523+0000","last_clean":"2024-09-13T18:36:36.889523+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:36:36.889523+0000","last_undegraded":"2024-09-13T18:36:36.889523+0000","last_fullsized":"2024-09-13T18:36:36.889523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664780+0000","last_change":"2024-09-13T18:36:34.635305+0000","last_active":"2024-09-13T18:36:38.664780+0000","last_peered":"2024-09-13T18:36:38.664780+0000","last_clean":"2024-09-13T18:36:38.664780+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:38.664780+0000","last_undegraded":"2024-09-13T18:36:38.664780+0000","last_fullsized":"2024-09-13T18:36:38.664780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:20.849745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657275+0000","last_change":"2024-09-13T18:36:36.656517+0000","last_active":"2024-09-13T18:36:36.657275+0000","last_peered":"2024-09-13T18:36:36.657275+0000","last_clean":"2024-09-13T18:36:36.657275+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:36.657275+0000","last_undegraded":"2024-09-13T18:36:36.657275+0000","last_fullsized":"2024-09-13T18:36:36.657275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:14.465685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.663921+0000","last_change":"2024-09-13T18:36:36.656192+0000","last_active":"2024-09-13T18:36:38.663921+0000","last_peered":"2024-09-13T18:36:38.663921+0000","last_clean":"2024-09-13T18:36:38.663921+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:38.663921+0000","last_undegraded":"2024-09-13T18:36:38.663921+0000","last_fullsized":"2024-09-13T18:36:38.663921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:00.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.658738+0000","last_change":"2024-09-13T18:36:36.657279+0000","last_active":"2024-09-13T18:36:36.658738+0000","last_peered":"2024-09-13T18:36:36.658738+0000","last_clean":"2024-09-13T18:36:36.658738+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:36.658738+0000","last_undegraded":"2024-09-13T18:36:36.658738+0000","last_fullsized":"2024-09-13T18:36:36.658738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:57:41.843736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657169+0000","last_change":"2024-09-13T18:36:36.656374+0000","last_active":"2024-09-13T18:36:36.657169+0000","last_peered":"2024-09-13T18:36:36.657169+0000","last_clean":"2024-09-13T18:36:36.657169+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:36.657169+0000","last_undegraded":"2024-09-13T18:36:36.657169+0000","last_fullsized":"2024-09-13T18:36:36.657169+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:43.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664734+0000","last_change":"2024-09-13T18:36:34.636948+0000","last_active":"2024-09-13T18:36:38.664734+0000","last_peered":"2024-09-13T18:36:38.664734+0000","last_clean":"2024-09-13T18:36:38.664734+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:38.664734+0000","last_undegraded":"2024-09-13T18:36:38.664734+0000","last_fullsized":"2024-09-13T18:36:38.664734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:35.345545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051587,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":2,"up_from":28,"seq":120259084291,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]}]},{"osd":1,"up_from":27,"seq":115964116995,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":868352,"data_stored":684391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]}]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:43.886 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:43.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.981+0000 7f694a8e5640 1 Processor -- start 2024-09-13T18:36:43.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.981+0000 7f694a8e5640 1 -- start start 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f694a8e5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944095650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f694a8e5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f694406cca0 0x7f6944095b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f694a8e5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 0x7f694406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f694a8e5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694406db30 con 0x7f694406cca0 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f694a8e5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694406e080 con 0x7f6944095270 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f694a8e5640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f694406e390 con 0x7f694406b1d0 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f69437fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f694406cca0 0x7f6944095b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6948e82640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 0x7f694406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6948e82640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 0x7f694406b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60706/0 (socket says 172.21.15.177:60706) 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f69437fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f694406cca0 0x7f6944095b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46038/0 (socket says 172.21.15.177:46038) 2024-09-13T18:36:43.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6948e82640 1 -- 172.21.15.177:0/4177537579 learned_addr learned my addr 172.21.15.177:0/4177537579 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944095650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57110/0 (socket says 172.21.15.177:57110) 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 -- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 msgr2=0x7f694406b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 --2- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 0x7f694406b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 -- 172.21.15.177:0/4177537579 >> v2:172.21.15.177:3300/0 conn(0x7f694406cca0 msgr2=0x7f6944095b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 --2- 172.21.15.177:0/4177537579 >> v2:172.21.15.177:3300/0 conn(0x7f694406cca0 0x7f6944095b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6943fff640 1 -- 172.21.15.177:0/4177537579 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694406bb80 con 0x7f6944095270 2024-09-13T18:36:43.986 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.982+0000 7f6948e82640 1 --2- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 0x7f694406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f6943fff640 1 --2- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944095650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f693400ba50 tx=0x7f693402c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40f49cf3f32f84f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f6942ffd640 1 -- 172.21.15.177:0/4177537579 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934032480 con 0x7f6944095270 2024-09-13T18:36:43.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f6942ffd640 1 -- 172.21.15.177:0/4177537579 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693402f030 con 0x7f6944095270 2024-09-13T18:36:43.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f6942ffd640 1 -- 172.21.15.177:0/4177537579 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934032800 con 0x7f6944095270 2024-09-13T18:36:43.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f694a8e5640 1 -- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 msgr2=0x7f6944095650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f694a8e5640 1 --2- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944095650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f693400ba50 tx=0x7f693402c690 comp rx=0 tx=0).stop 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f694a8e5640 1 -- 172.21.15.177:0/4177537579 shutdown_connections 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f694a8e5640 1 --2- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406b1d0 0x7f694406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f694a8e5640 1 --2- 172.21.15.177:0/4177537579 >> v2:172.21.15.177:3300/0 conn(0x7f694406cca0 0x7f6944095b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.983+0000 7f694a8e5640 1 --2- 172.21.15.177:0/4177537579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944095650 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.984+0000 7f694a8e5640 1 -- 172.21.15.177:0/4177537579 >> 172.21.15.177:0/4177537579 conn(0x7f6944098ba0 msgr2=0x7f6944096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.984+0000 7f694a8e5640 1 -- 172.21.15.177:0/4177537579 shutdown_connections 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.984+0000 7f694a8e5640 1 -- 172.21.15.177:0/4177537579 wait complete. 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.984+0000 7f694a8e5640 1 Processor -- start 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 -- start start 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 0x7f6944093260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406cca0 0x7f69440937a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944093ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69441253b0 con 0x7f694406b1d0 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69441264e0 con 0x7f6944095270 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f694a8e5640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6944125ed0 con 0x7f694406cca0 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 0x7f6944093260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 0x7f6944093260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46054/0 (socket says 172.21.15.177:46054) 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 -- 172.21.15.177:0/2255921974 learned_addr learned my addr 172.21.15.177:0/2255921974 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f69437fe640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406cca0 0x7f69440937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6948e82640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944093ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 -- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406cca0 msgr2=0x7f69440937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406cca0 0x7f69440937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 -- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 msgr2=0x7f6944093ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944093ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:43.991 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6943fff640 1 -- 172.21.15.177:0/2255921974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c009b30 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.985+0000 7f6948e82640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944093ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.986+0000 7f6943fff640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 0x7f6944093260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f693400ba50 tx=0x7f6934004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.986+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934035620 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.986+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6934035780 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.986+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693400b6b0 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.986+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934035b30 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.986+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694412a530 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.987+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f69340493b0 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.987+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694406b640 con 0x7f694406b1d0 2024-09-13T18:36:43.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.987+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f693407cf60 con 0x7f694406b1d0 2024-09-13T18:36:43.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:43.991+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f693407c010 con 0x7f694406b1d0 2024-09-13T18:36:44.123 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.121+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6944095790 con 0x7f6930047560 2024-09-13T18:36:44.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.122+0000 7f6940ff9640 1 -- 172.21.15.177:0/2255921974 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34788 (unknown 2965378022 0 3289417995) 0x7f6944095790 con 0x7f6930047560 2024-09-13T18:36:44.125 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:44.125 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 >> v1:172.21.15.177:6816/516212916 conn(0x7f6930047560 legacy=0x7f6930049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 msgr2=0x7f6944093260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 0x7f6944093260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f693400ba50 tx=0x7f6934004730 comp rx=0 tx=0).stop 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 shutdown_connections 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6944095270 0x7f6944093ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f694406cca0 0x7f69440937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 --2- 172.21.15.177:0/2255921974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f694406b1d0 0x7f6944093260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 >> 172.21.15.177:0/2255921974 conn(0x7f6944098ba0 msgr2=0x7f6944099950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 shutdown_connections 2024-09-13T18:36:44.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.124+0000 7f694a8e5640 1 -- 172.21.15.177:0/2255921974 wait complete. 2024-09-13T18:36:44.136 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-13T18:36:42.593773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112316,"kb_used_data":2244,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375044,"statfs":{"total":386547056640,"available":386432045056,"internally_reserved":0,"allocated":2297856,"data_stored":1566874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.941701"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889716+0000","last_change":"2024-09-13T18:36:36.889220+0000","last_active":"2024-09-13T18:36:36.889716+0000","last_peered":"2024-09-13T18:36:36.889716+0000","last_clean":"2024-09-13T18:36:36.889716+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:36:36.889716+0000","last_undegraded":"2024-09-13T18:36:36.889716+0000","last_fullsized":"2024-09-13T18:36:36.889716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:19.625333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664807+0000","last_change":"2024-09-13T18:36:34.636621+0000","last_active":"2024-09-13T18:36:38.664807+0000","last_peered":"2024-09-13T18:36:38.664807+0000","last_clean":"2024-09-13T18:36:38.664807+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:38.664807+0000","last_undegraded":"2024-09-13T18:36:38.664807+0000","last_fullsized":"2024-09-13T18:36:38.664807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:09.196966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889523+0000","last_change":"2024-09-13T18:36:36.888348+0000","last_active":"2024-09-13T18:36:36.889523+0000","last_peered":"2024-09-13T18:36:36.889523+0000","last_clean":"2024-09-13T18:36:36.889523+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:36:36.889523+0000","last_undegraded":"2024-09-13T18:36:36.889523+0000","last_fullsized":"2024-09-13T18:36:36.889523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664780+0000","last_change":"2024-09-13T18:36:34.635305+0000","last_active":"2024-09-13T18:36:38.664780+0000","last_peered":"2024-09-13T18:36:38.664780+0000","last_clean":"2024-09-13T18:36:38.664780+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:38.664780+0000","last_undegraded":"2024-09-13T18:36:38.664780+0000","last_fullsized":"2024-09-13T18:36:38.664780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:20.849745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657275+0000","last_change":"2024-09-13T18:36:36.656517+0000","last_active":"2024-09-13T18:36:36.657275+0000","last_peered":"2024-09-13T18:36:36.657275+0000","last_clean":"2024-09-13T18:36:36.657275+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:36.657275+0000","last_undegraded":"2024-09-13T18:36:36.657275+0000","last_fullsized":"2024-09-13T18:36:36.657275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:14.465685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.663921+0000","last_change":"2024-09-13T18:36:36.656192+0000","last_active":"2024-09-13T18:36:38.663921+0000","last_peered":"2024-09-13T18:36:38.663921+0000","last_clean":"2024-09-13T18:36:38.663921+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:38.663921+0000","last_undegraded":"2024-09-13T18:36:38.663921+0000","last_fullsized":"2024-09-13T18:36:38.663921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:00.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.658738+0000","last_change":"2024-09-13T18:36:36.657279+0000","last_active":"2024-09-13T18:36:36.658738+0000","last_peered":"2024-09-13T18:36:36.658738+0000","last_clean":"2024-09-13T18:36:36.658738+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:36.658738+0000","last_undegraded":"2024-09-13T18:36:36.658738+0000","last_fullsized":"2024-09-13T18:36:36.658738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:57:41.843736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657169+0000","last_change":"2024-09-13T18:36:36.656374+0000","last_active":"2024-09-13T18:36:36.657169+0000","last_peered":"2024-09-13T18:36:36.657169+0000","last_clean":"2024-09-13T18:36:36.657169+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:36.657169+0000","last_undegraded":"2024-09-13T18:36:36.657169+0000","last_fullsized":"2024-09-13T18:36:36.657169+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:43.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664734+0000","last_change":"2024-09-13T18:36:34.636948+0000","last_active":"2024-09-13T18:36:38.664734+0000","last_peered":"2024-09-13T18:36:38.664734+0000","last_clean":"2024-09-13T18:36:38.664734+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:38.664734+0000","last_undegraded":"2024-09-13T18:36:38.664734+0000","last_fullsized":"2024-09-13T18:36:38.664734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:35.345545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051587,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":2,"up_from":28,"seq":120259084291,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]}]},{"osd":1,"up_from":27,"seq":115964116995,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":868352,"data_stored":684391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]}]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:44.137 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:36:44.137 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T18:36:44.138 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T18:36:44.138 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 Processor -- start 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 -- start start 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 0x7f141c12a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c06e090 0x7f141c06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 0x7f141c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141c12a2f0 con 0x7f141c12adf0 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c097e00 con 0x7f141c12a4d0 2024-09-13T18:36:44.235 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f1421b45640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f141c06e870 con 0x7f141c06e090 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 0x7f141c12a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141bfff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 0x7f141c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141affd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c06e090 0x7f141c06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 0x7f141c12a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57122/0 (socket says 172.21.15.177:57122) 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141affd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c06e090 0x7f141c06c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60730/0 (socket says 172.21.15.177:60730) 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141bfff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 0x7f141c06c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46058/0 (socket says 172.21.15.177:46058) 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.232+0000 7f141b7fe640 1 -- 172.21.15.177:0/75468373 learned_addr learned my addr 172.21.15.177:0/75468373 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141b7fe640 1 -- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c06e090 msgr2=0x7f141c06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141b7fe640 1 --2- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c06e090 0x7f141c06c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141b7fe640 1 -- 172.21.15.177:0/75468373 >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 msgr2=0x7f141c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141b7fe640 1 --2- 172.21.15.177:0/75468373 >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 0x7f141c06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141b7fe640 1 -- 172.21.15.177:0/75468373 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c06cd60 con 0x7f141c12a4d0 2024-09-13T18:36:44.237 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141bfff640 1 --2- 172.21.15.177:0/75468373 >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 0x7f141c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.233+0000 7f141b7fe640 1 --2- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 0x7f141c12a8b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f140c009870 tx=0x7f140c02a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc243fcd8f23fbca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:44.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f141a7fc640 1 -- 172.21.15.177:0/75468373 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c02adf0 con 0x7f141c12a4d0 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f141a7fc640 1 -- 172.21.15.177:0/75468373 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c032440 con 0x7f141c12a4d0 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f141a7fc640 1 -- 172.21.15.177:0/75468373 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0327d0 con 0x7f141c12a4d0 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 -- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 msgr2=0x7f141c12a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 --2- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 0x7f141c12a8b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f140c009870 tx=0x7f140c02a0a0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 -- 172.21.15.177:0/75468373 shutdown_connections 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 --2- 172.21.15.177:0/75468373 >> v2:172.21.15.177:3300/0 conn(0x7f141c12adf0 0x7f141c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 --2- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c06e090 0x7f141c06c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 --2- 172.21.15.177:0/75468373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12a4d0 0x7f141c12a8b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 -- 172.21.15.177:0/75468373 >> 172.21.15.177:0/75468373 conn(0x7f141c098ba0 msgr2=0x7f141c096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 -- 172.21.15.177:0/75468373 shutdown_connections 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.234+0000 7f1421b45640 1 -- 172.21.15.177:0/75468373 wait complete. 2024-09-13T18:36:44.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.235+0000 7f1421b45640 1 Processor -- start 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.235+0000 7f1421b45640 1 -- start start 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 0x7f141c08b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c12a4d0 0x7f141c08b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12adf0 0x7f141c1be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c08d0e0 con 0x7f141c06e090 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c127110 con 0x7f141c12adf0 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f141c08d550 con 0x7f141c12a4d0 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 0x7f141c08b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141affd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c12a4d0 0x7f141c08b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 0x7f141c08b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46064/0 (socket says 172.21.15.177:46064) 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 -- 172.21.15.177:0/3736936645 learned_addr learned my addr 172.21.15.177:0/3736936645 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141bfff640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12adf0 0x7f141c1be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 -- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c12a4d0 msgr2=0x7f141c08b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c12a4d0 0x7f141c08b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 -- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12adf0 msgr2=0x7f141c1be3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12adf0 0x7f141c1be3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 -- 172.21.15.177:0/3736936645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414009a70 con 0x7f141c06e090 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141affd640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c12a4d0 0x7f141c08b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f141b7fe640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 0x7f141c08b320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c03d830 con 0x7f141c06e090 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c03d9b0 con 0x7f141c06e090 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c03dd60 con 0x7f141c06e090 2024-09-13T18:36:44.242 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c0094d0 con 0x7f141c06e090 2024-09-13T18:36:44.243 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.236+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c08cd70 con 0x7f141c06e090 2024-09-13T18:36:44.243 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.237+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f140c039040 con 0x7f141c06e090 2024-09-13T18:36:44.243 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.237+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c12a8b0 con 0x7f141c06e090 2024-09-13T18:36:44.243 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.238+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f140c07d2f0 con 0x7f141c06e090 2024-09-13T18:36:44.243 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.241+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f140c05e990 con 0x7f141c06e090 2024-09-13T18:36:44.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.397+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f141c0710b0 con 0x7f141c06e090 2024-09-13T18:36:44.400 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.398+0000 7f1413fff640 1 -- 172.21.15.177:0/3736936645 <== mon.0 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f140c037070 con 0x7f141c06e090 2024-09-13T18:36:44.400 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:44.400 INFO:teuthology.orchestra.run.smithi177.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 >> v1:172.21.15.177:6816/516212916 conn(0x7f13fc047560 legacy=0x7f13fc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 msgr2=0x7f141c08b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 0x7f141c08b320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 shutdown_connections 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f141c12adf0 0x7f141c1be3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f141c12a4d0 0x7f141c08b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.399+0000 7f1421b45640 1 --2- 172.21.15.177:0/3736936645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141c06e090 0x7f141c08b320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.400+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 >> 172.21.15.177:0/3736936645 conn(0x7f141c098ba0 msgr2=0x7f141c06ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.400+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 shutdown_connections 2024-09-13T18:36:44.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.400+0000 7f1421b45640 1 -- 172.21.15.177:0/3736936645 wait complete. 2024-09-13T18:36:44.412 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T18:36:44.413 INFO:teuthology.run_tasks:Running task print... 2024-09-13T18:36:44.423 INFO:teuthology.task.print:**** done ceph.restart 2024-09-13T18:36:44.423 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-09-13T18:36:44.439 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-09-13T18:36:44.452 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2024-09-13T18:36:44.467 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T18:36:44.483 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:36:44.483 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:44.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 Processor -- start 2024-09-13T18:36:44.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 -- start start 2024-09-13T18:36:44.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d2c06c640 0x7f9d2c06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c12a8d0 0x7f9d2c132500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c06c460 con 0x7f9d2c06cf60 2024-09-13T18:36:44.581 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c097e00 con 0x7f9d2c06c640 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.578+0000 7f9d32c7d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c06d770 con 0x7f9d2c12a8d0 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d3121a640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c12a8d0 0x7f9d2c132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d23fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f9d2c06cf60 0x7f9d2c12a1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42720/0 (socket says 172.21.15.177:42720) 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d30a19640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d2c06c640 0x7f9d2c06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d23fff640 1 -- 172.21.15.177:0/2617533234 learned_addr learned my addr 172.21.15.177:0/2617533234 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d30a19640 1 -- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c12a8d0 msgr2=0x7f9d2c132500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d30a19640 1 --2- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c12a8d0 0x7f9d2c132500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d30a19640 1 -- 172.21.15.177:0/2617533234 >> v1:172.21.15.177:6789/0 conn(0x7f9d2c06cf60 legacy=0x7f9d2c12a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d30a19640 1 -- 172.21.15.177:0/2617533234 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c06e0b0 con 0x7f9d2c06c640 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d237fe640 1 -- 172.21.15.177:0/2617533234 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848370950 0 0) 0x7f9d2c06c460 con 0x7f9d2c06cf60 2024-09-13T18:36:44.582 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.579+0000 7f9d3121a640 1 --2- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c12a8d0 0x7f9d2c132500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d30a19640 1 --2- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06c640 0x7f9d2c06ca20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009dc0 tx=0x7f9d2802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5cad60b386359e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d237fe640 1 -- 172.21.15.177:0/2617533234 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28007740 con 0x7f9d2c06c640 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d237fe640 1 -- 172.21.15.177:0/2617533234 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2802d050 con 0x7f9d2c06c640 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d237fe640 1 -- 172.21.15.177:0/2617533234 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28007ac0 con 0x7f9d2c06c640 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06c640 msgr2=0x7f9d2c06ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06c640 0x7f9d2c06ca20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009dc0 tx=0x7f9d2802a5f0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2617533234 shutdown_connections 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c12a8d0 0x7f9d2c132500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2617533234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06c640 0x7f9d2c06ca20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2617533234 >> 172.21.15.177:0/2617533234 conn(0x7f9d2c098ba0 msgr2=0x7f9d2c096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2617533234 shutdown_connections 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.580+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2617533234 wait complete. 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.581+0000 7f9d32c7d640 1 Processor -- start 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.581+0000 7f9d32c7d640 1 -- start start 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d32c7d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 0x7f9d2c127fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d32c7d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 0x7f9d2c128500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d32c7d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2c12a8d0 0x7f9d2c128a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d32c7d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c122900 con 0x7f9d2c06cf60 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d32c7d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c123bd0 con 0x7f9d2c12a8d0 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d32c7d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c124bf0 con 0x7f9d2c06c640 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 0x7f9d2c128500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.586 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d30a19640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 0x7f9d2c127fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d3121a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2c12a8d0 0x7f9d2c128a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d30a19640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 0x7f9d2c127fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60742/0 (socket says 172.21.15.177:60742) 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d30a19640 1 -- 172.21.15.177:0/2788931588 learned_addr learned my addr 172.21.15.177:0/2788931588 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 0x7f9d2c128500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46074/0 (socket says 172.21.15.177:46074) 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 -- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 msgr2=0x7f9d2c127fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 0x7f9d2c127fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 -- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2c12a8d0 msgr2=0x7f9d2c128a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2c12a8d0 0x7f9d2c128a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 -- 172.21.15.177:0/2788931588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24009400 con 0x7f9d2c06cf60 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d30a19640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 0x7f9d2c127fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d3121a640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2c12a8d0 0x7f9d2c128a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.582+0000 7f9d23fff640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 0x7f9d2c128500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c3e0 tx=0x7f9d1c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:44.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.583+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0074e0 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.583+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c007640 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.583+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d28009a70 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.583+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c004a20 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.583+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c1299c0 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.584+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c1329c0 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.584+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9d1c004b80 con 0x7f9d2c06cf60 2024-09-13T18:36:44.588 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.584+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f9d1c05c590 con 0x7f9d2c06cf60 2024-09-13T18:36:44.589 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.587+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f9d1c03a200 con 0x7f9d2c06cf60 2024-09-13T18:36:44.725 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.723+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d2c06ca20 con 0x7f9d18047590 2024-09-13T18:36:44.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.724+0000 7f9d217fa640 1 -- 172.21.15.177:0/2788931588 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34776 (unknown 2965378022 0 3076606257) 0x7f9d2c06ca20 con 0x7f9d18047590 2024-09-13T18:36:44.727 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:44.727 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:44.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 >> v1:172.21.15.177:6816/516212916 conn(0x7f9d18047590 legacy=0x7f9d18049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 msgr2=0x7f9d2c128500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 0x7f9d2c128500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c3e0 tx=0x7f9d1c00c7e0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 shutdown_connections 2024-09-13T18:36:44.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2c12a8d0 0x7f9d2c128a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d2c06cf60 0x7f9d2c128500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 --2- 172.21.15.177:0/2788931588 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f9d2c06c640 0x7f9d2c127fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 >> 172.21.15.177:0/2788931588 conn(0x7f9d2c098ba0 msgr2=0x7f9d2c095270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 shutdown_connections 2024-09-13T18:36:44.730 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.726+0000 7f9d32c7d640 1 -- 172.21.15.177:0/2788931588 wait complete. 2024-09-13T18:36:44.739 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-13T18:36:44.594061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112328,"kb_used_data":2252,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375032,"statfs":{"total":386547056640,"available":386432032768,"internally_reserved":0,"allocated":2306048,"data_stored":1570439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.937585"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889716+0000","last_change":"2024-09-13T18:36:36.889220+0000","last_active":"2024-09-13T18:36:36.889716+0000","last_peered":"2024-09-13T18:36:36.889716+0000","last_clean":"2024-09-13T18:36:36.889716+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:36:36.889716+0000","last_undegraded":"2024-09-13T18:36:36.889716+0000","last_fullsized":"2024-09-13T18:36:36.889716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:19.625333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664807+0000","last_change":"2024-09-13T18:36:34.636621+0000","last_active":"2024-09-13T18:36:38.664807+0000","last_peered":"2024-09-13T18:36:38.664807+0000","last_clean":"2024-09-13T18:36:38.664807+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:38.664807+0000","last_undegraded":"2024-09-13T18:36:38.664807+0000","last_fullsized":"2024-09-13T18:36:38.664807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:09.196966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889523+0000","last_change":"2024-09-13T18:36:36.888348+0000","last_active":"2024-09-13T18:36:36.889523+0000","last_peered":"2024-09-13T18:36:36.889523+0000","last_clean":"2024-09-13T18:36:36.889523+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:36:36.889523+0000","last_undegraded":"2024-09-13T18:36:36.889523+0000","last_fullsized":"2024-09-13T18:36:36.889523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664780+0000","last_change":"2024-09-13T18:36:34.635305+0000","last_active":"2024-09-13T18:36:38.664780+0000","last_peered":"2024-09-13T18:36:38.664780+0000","last_clean":"2024-09-13T18:36:38.664780+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:38.664780+0000","last_undegraded":"2024-09-13T18:36:38.664780+0000","last_fullsized":"2024-09-13T18:36:38.664780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:20.849745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657275+0000","last_change":"2024-09-13T18:36:36.656517+0000","last_active":"2024-09-13T18:36:36.657275+0000","last_peered":"2024-09-13T18:36:36.657275+0000","last_clean":"2024-09-13T18:36:36.657275+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:36.657275+0000","last_undegraded":"2024-09-13T18:36:36.657275+0000","last_fullsized":"2024-09-13T18:36:36.657275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:14.465685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.663921+0000","last_change":"2024-09-13T18:36:36.656192+0000","last_active":"2024-09-13T18:36:38.663921+0000","last_peered":"2024-09-13T18:36:38.663921+0000","last_clean":"2024-09-13T18:36:38.663921+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:38.663921+0000","last_undegraded":"2024-09-13T18:36:38.663921+0000","last_fullsized":"2024-09-13T18:36:38.663921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:00.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.658738+0000","last_change":"2024-09-13T18:36:36.657279+0000","last_active":"2024-09-13T18:36:36.658738+0000","last_peered":"2024-09-13T18:36:36.658738+0000","last_clean":"2024-09-13T18:36:36.658738+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:36.658738+0000","last_undegraded":"2024-09-13T18:36:36.658738+0000","last_fullsized":"2024-09-13T18:36:36.658738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:57:41.843736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657169+0000","last_change":"2024-09-13T18:36:36.656374+0000","last_active":"2024-09-13T18:36:36.657169+0000","last_peered":"2024-09-13T18:36:36.657169+0000","last_clean":"2024-09-13T18:36:36.657169+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:36.657169+0000","last_undegraded":"2024-09-13T18:36:36.657169+0000","last_fullsized":"2024-09-13T18:36:36.657169+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:43.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664734+0000","last_change":"2024-09-13T18:36:34.636948+0000","last_active":"2024-09-13T18:36:38.664734+0000","last_peered":"2024-09-13T18:36:38.664734+0000","last_clean":"2024-09-13T18:36:38.664734+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:38.664734+0000","last_undegraded":"2024-09-13T18:36:38.664734+0000","last_fullsized":"2024-09-13T18:36:38.664734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:35.345545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051587,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":2,"up_from":28,"seq":120259084292,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":27,"seq":115964116996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":856,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":876544,"data_stored":687956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:44.740 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:44.837 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.835+0000 7f1d7457d640 1 Processor -- start 2024-09-13T18:36:44.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.835+0000 7f1d7457d640 1 -- start start 2024-09-13T18:36:44.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d7457d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d6c06b440 0x7f1d6c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d7457d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 0x7f1d6c12d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d7457d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d7457d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c06d840 con 0x7f1d6c06b440 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d7457d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c06c4a0 con 0x7f1d6c12da20 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d7457d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c06db60 con 0x7f1d6c06bd60 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d71b18640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 0x7f1d6c12d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72319640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d6c06b440 0x7f1d6c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72b1a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d71b18640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 0x7f1d6c12d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60754/0 (socket says 172.21.15.177:60754) 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d71b18640 1 -- 172.21.15.177:0/3579250557 learned_addr learned my addr 172.21.15.177:0/3579250557 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72b1a640 1 -- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 msgr2=0x7f1d6c12d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72b1a640 1 --2- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 0x7f1d6c12d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72b1a640 1 -- 172.21.15.177:0/3579250557 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c06b440 msgr2=0x7f1d6c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:44.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72b1a640 1 --2- 172.21.15.177:0/3579250557 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c06b440 0x7f1d6c06b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72b1a640 1 -- 172.21.15.177:0/3579250557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c06c170 con 0x7f1d6c12da20 2024-09-13T18:36:44.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d72319640 1 --2- 172.21.15.177:0/3579250557 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c06b440 0x7f1d6c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.836+0000 7f1d71b18640 1 --2- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 0x7f1d6c12d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.837+0000 7f1d72b1a640 1 --2- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12deb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009d90 tx=0x7f1d6402a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4304aa5c42879def server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.837+0000 7f1d71317640 1 -- 172.21.15.177:0/3579250557 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64007740 con 0x7f1d6c12da20 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.837+0000 7f1d71317640 1 -- 172.21.15.177:0/3579250557 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6402d050 con 0x7f1d6c12da20 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.837+0000 7f1d71317640 1 -- 172.21.15.177:0/3579250557 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64007ac0 con 0x7f1d6c12da20 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 -- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 msgr2=0x7f1d6c12deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 --2- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12deb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009d90 tx=0x7f1d6402a490 comp rx=0 tx=0).stop 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 -- 172.21.15.177:0/3579250557 shutdown_connections 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 --2- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12deb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 --2- 172.21.15.177:0/3579250557 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06bd60 0x7f1d6c12d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 --2- 172.21.15.177:0/3579250557 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c06b440 0x7f1d6c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 -- 172.21.15.177:0/3579250557 >> 172.21.15.177:0/3579250557 conn(0x7f1d6c09fe10 msgr2=0x7f1d6c0a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 -- 172.21.15.177:0/3579250557 shutdown_connections 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.838+0000 7f1d7457d640 1 -- 172.21.15.177:0/3579250557 wait complete. 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 Processor -- start 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 -- start start 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 0x7f1d6c12e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c06bd60 0x7f1d6c12ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c130540 con 0x7f1d6c06bd60 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1312a0 con 0x7f1d6c12da20 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d7457d640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1301e0 con 0x7f1d6c06b440 2024-09-13T18:36:44.848 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d72319640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 0x7f1d6c12e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d71b18640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c06bd60 0x7f1d6c12ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d72319640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 0x7f1d6c12e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60770/0 (socket says 172.21.15.177:60770) 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d72319640 1 -- 172.21.15.177:0/1777099621 learned_addr learned my addr 172.21.15.177:0/1777099621 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d72b1a640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d71b18640 1 -- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 msgr2=0x7f1d6c12e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d71b18640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 0x7f1d6c12e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d71b18640 1 -- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 msgr2=0x7f1d6c12efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d71b18640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d71b18640 1 -- 172.21.15.177:0/1777099621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009400 con 0x7f1d6c06bd60 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.839+0000 7f1d72319640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 0x7f1d6c12e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.840+0000 7f1d71b18640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c06bd60 0x7f1d6c12ea80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e900 tx=0x7f1d5c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.840+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00ca00 con 0x7f1d6c06bd60 2024-09-13T18:36:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.840+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c00cb60 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.840+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c0094d0 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.840+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004030 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.840+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64009a70 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.841+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c12d4e0 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.841+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1d5c004190 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.841+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f1d5c05c880 con 0x7f1d6c06bd60 2024-09-13T18:36:44.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.844+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f1d5c03a4f0 con 0x7f1d6c06bd60 2024-09-13T18:36:44.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.977+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d6c12de30 con 0x7f1d50047590 2024-09-13T18:36:44.980 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.978+0000 7f1d62ffd640 1 -- 172.21.15.177:0/1777099621 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34776 (unknown 2965378022 0 3076606257) 0x7f1d6c12de30 con 0x7f1d50047590 2024-09-13T18:36:44.981 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:44.981 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 >> v1:172.21.15.177:6816/516212916 conn(0x7f1d50047590 legacy=0x7f1d50049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c06bd60 msgr2=0x7f1d6c12ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c06bd60 0x7f1d6c12ea80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e900 tx=0x7f1d5c00edc0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 shutdown_connections 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c12da20 0x7f1d6c12efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c06bd60 0x7f1d6c12ea80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 --2- 172.21.15.177:0/1777099621 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f1d6c06b440 0x7f1d6c12e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:44.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 >> 172.21.15.177:0/1777099621 conn(0x7f1d6c09fe10 msgr2=0x7f1d6c12cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:44.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 shutdown_connections 2024-09-13T18:36:44.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:44.980+0000 7f1d7457d640 1 -- 172.21.15.177:0/1777099621 wait complete. 2024-09-13T18:36:44.992 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-13T18:36:44.594061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112328,"kb_used_data":2252,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375032,"statfs":{"total":386547056640,"available":386432032768,"internally_reserved":0,"allocated":2306048,"data_stored":1570439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.937585"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889716+0000","last_change":"2024-09-13T18:36:36.889220+0000","last_active":"2024-09-13T18:36:36.889716+0000","last_peered":"2024-09-13T18:36:36.889716+0000","last_clean":"2024-09-13T18:36:36.889716+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:36:36.889716+0000","last_undegraded":"2024-09-13T18:36:36.889716+0000","last_fullsized":"2024-09-13T18:36:36.889716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:19.625333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664807+0000","last_change":"2024-09-13T18:36:34.636621+0000","last_active":"2024-09-13T18:36:38.664807+0000","last_peered":"2024-09-13T18:36:38.664807+0000","last_clean":"2024-09-13T18:36:38.664807+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:38.664807+0000","last_undegraded":"2024-09-13T18:36:38.664807+0000","last_fullsized":"2024-09-13T18:36:38.664807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:09.196966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889523+0000","last_change":"2024-09-13T18:36:36.888348+0000","last_active":"2024-09-13T18:36:36.889523+0000","last_peered":"2024-09-13T18:36:36.889523+0000","last_clean":"2024-09-13T18:36:36.889523+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:36:36.889523+0000","last_undegraded":"2024-09-13T18:36:36.889523+0000","last_fullsized":"2024-09-13T18:36:36.889523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664780+0000","last_change":"2024-09-13T18:36:34.635305+0000","last_active":"2024-09-13T18:36:38.664780+0000","last_peered":"2024-09-13T18:36:38.664780+0000","last_clean":"2024-09-13T18:36:38.664780+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:38.664780+0000","last_undegraded":"2024-09-13T18:36:38.664780+0000","last_fullsized":"2024-09-13T18:36:38.664780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:20.849745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657275+0000","last_change":"2024-09-13T18:36:36.656517+0000","last_active":"2024-09-13T18:36:36.657275+0000","last_peered":"2024-09-13T18:36:36.657275+0000","last_clean":"2024-09-13T18:36:36.657275+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:36.657275+0000","last_undegraded":"2024-09-13T18:36:36.657275+0000","last_fullsized":"2024-09-13T18:36:36.657275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:14.465685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.663921+0000","last_change":"2024-09-13T18:36:36.656192+0000","last_active":"2024-09-13T18:36:38.663921+0000","last_peered":"2024-09-13T18:36:38.663921+0000","last_clean":"2024-09-13T18:36:38.663921+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:38.663921+0000","last_undegraded":"2024-09-13T18:36:38.663921+0000","last_fullsized":"2024-09-13T18:36:38.663921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:00.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.658738+0000","last_change":"2024-09-13T18:36:36.657279+0000","last_active":"2024-09-13T18:36:36.658738+0000","last_peered":"2024-09-13T18:36:36.658738+0000","last_clean":"2024-09-13T18:36:36.658738+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:36.658738+0000","last_undegraded":"2024-09-13T18:36:36.658738+0000","last_fullsized":"2024-09-13T18:36:36.658738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:57:41.843736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657169+0000","last_change":"2024-09-13T18:36:36.656374+0000","last_active":"2024-09-13T18:36:36.657169+0000","last_peered":"2024-09-13T18:36:36.657169+0000","last_clean":"2024-09-13T18:36:36.657169+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:36.657169+0000","last_undegraded":"2024-09-13T18:36:36.657169+0000","last_fullsized":"2024-09-13T18:36:36.657169+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:43.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664734+0000","last_change":"2024-09-13T18:36:34.636948+0000","last_active":"2024-09-13T18:36:38.664734+0000","last_peered":"2024-09-13T18:36:38.664734+0000","last_clean":"2024-09-13T18:36:38.664734+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:38.664734+0000","last_undegraded":"2024-09-13T18:36:38.664734+0000","last_fullsized":"2024-09-13T18:36:38.664734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:35.345545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051587,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":2,"up_from":28,"seq":120259084292,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":27,"seq":115964116996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":856,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":876544,"data_stored":687956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:44.997 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:36:44.997 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 Processor -- start 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 -- start start 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fde0006c640 0x7fde0006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 0x7fde00132500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0006c460 con 0x7fde0006cf60 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde00097e00 con 0x7fde0006c640 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fde05f1f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde0006d770 con 0x7fde0012a8d0 2024-09-13T18:36:45.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fddfffff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 0x7fde00132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fddff7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fde0006c640 0x7fde0006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fddfffff640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 0x7fde00132500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60784/0 (socket says 172.21.15.177:60784) 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fddfffff640 1 -- 172.21.15.177:0/1603060560 learned_addr learned my addr 172.21.15.177:0/1603060560 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.092+0000 7fddfeffd640 1 --1- 172.21.15.177:0/1603060560 >> v1:172.21.15.177:6789/0 conn(0x7fde0006cf60 0x7fde0012a1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42732/0 (socket says 172.21.15.177:42732) 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.093+0000 7fddff7fe640 1 -- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 msgr2=0x7fde00132500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.093+0000 7fddff7fe640 1 --2- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 0x7fde00132500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.093+0000 7fddff7fe640 1 -- 172.21.15.177:0/1603060560 >> v1:172.21.15.177:6789/0 conn(0x7fde0006cf60 legacy=0x7fde0012a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.093+0000 7fddff7fe640 1 -- 172.21.15.177:0/1603060560 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0006e0b0 con 0x7fde0006c640 2024-09-13T18:36:45.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.093+0000 7fddfffff640 1 --2- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 0x7fde00132500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:45.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.093+0000 7fddff7fe640 1 --2- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0006c640 0x7fde0006ca20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009dc0 tx=0x7fddf002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395a9a47f1242ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:45.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fddfe7fc640 1 -- 172.21.15.177:0/1603060560 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0007740 con 0x7fde0006c640 2024-09-13T18:36:45.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fddfe7fc640 1 -- 172.21.15.177:0/1603060560 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf002d050 con 0x7fde0006c640 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fddfe7fc640 1 -- 172.21.15.177:0/1603060560 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0007ac0 con 0x7fde0006c640 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 -- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0006c640 msgr2=0x7fde0006ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 --2- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0006c640 0x7fde0006ca20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009dc0 tx=0x7fddf002a5f0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 -- 172.21.15.177:0/1603060560 shutdown_connections 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 --2- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0012a8d0 0x7fde00132500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 --2- 172.21.15.177:0/1603060560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0006c640 0x7fde0006ca20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 -- 172.21.15.177:0/1603060560 >> 172.21.15.177:0/1603060560 conn(0x7fde00098ba0 msgr2=0x7fde00096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 -- 172.21.15.177:0/1603060560 shutdown_connections 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.094+0000 7fde05f1f640 1 -- 172.21.15.177:0/1603060560 wait complete. 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 Processor -- start 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 -- start start 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde0006c640 0x7fde00131680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 0x7fde00131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 0x7fde00132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0012d0f0 con 0x7fde0012a8d0 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.095+0000 7fde05f1f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0012c2f0 con 0x7fde0006c640 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fde05f1f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde0006b7b0 con 0x7fde0006cf60 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 0x7fde00132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 0x7fde00131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 0x7fde00131bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60788/0 (socket says 172.21.15.177:60788) 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 0x7fde00132100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46094/0 (socket says 172.21.15.177:46094) 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 -- 172.21.15.177:0/2917304756 learned_addr learned my addr 172.21.15.177:0/2917304756 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddff7fe640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde0006c640 0x7fde00131680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 -- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde0006c640 msgr2=0x7fde00131680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde0006c640 0x7fde00131680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 -- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 msgr2=0x7fde00132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 0x7fde00132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 -- 172.21.15.177:0/2917304756 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf80094d0 con 0x7fde0006cf60 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfffff640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 0x7fde00132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fddfeffd640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 0x7fde00131bc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fddf400c530 tx=0x7fddf400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4007480 con 0x7fde0006cf60 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf40075e0 con 0x7fde0006cf60 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40049e0 con 0x7fde0006cf60 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf0009a70 con 0x7fde0006cf60 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.096+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0012c540 con 0x7fde0006cf60 2024-09-13T18:36:45.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.097+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fddf4004b40 con 0x7fde0006cf60 2024-09-13T18:36:45.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.097+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0006ca20 con 0x7fde0006cf60 2024-09-13T18:36:45.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.098+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fddf405ba40 con 0x7fde0006cf60 2024-09-13T18:36:45.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.101+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fddf405de60 con 0x7fde0006cf60 2024-09-13T18:36:45.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.234+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde00132a40 con 0x7fddec047560 2024-09-13T18:36:45.237 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.235+0000 7fdde7fff640 1 -- 172.21.15.177:0/2917304756 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34776 (unknown 2965378022 0 3076606257) 0x7fddf0007540 con 0x7fddec047560 2024-09-13T18:36:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:45.239 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:45.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 >> v1:172.21.15.177:6816/516212916 conn(0x7fddec047560 legacy=0x7fddec049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 msgr2=0x7fde00131bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 0x7fde00131bc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fddf400c530 tx=0x7fddf400c9f0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.240 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 shutdown_connections 2024-09-13T18:36:45.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde0012a8d0 0x7fde00132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fde0006cf60 0x7fde00131bc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 --2- 172.21.15.177:0/2917304756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde0006c640 0x7fde00131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 >> 172.21.15.177:0/2917304756 conn(0x7fde00098ba0 msgr2=0x7fde00126d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:45.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 shutdown_connections 2024-09-13T18:36:45.241 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.237+0000 7fde05f1f640 1 -- 172.21.15.177:0/2917304756 wait complete. 2024-09-13T18:36:45.252 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-13T18:36:44.594061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112328,"kb_used_data":2252,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375032,"statfs":{"total":386547056640,"available":386432032768,"internally_reserved":0,"allocated":2306048,"data_stored":1570439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.937585"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889716+0000","last_change":"2024-09-13T18:36:36.889220+0000","last_active":"2024-09-13T18:36:36.889716+0000","last_peered":"2024-09-13T18:36:36.889716+0000","last_clean":"2024-09-13T18:36:36.889716+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:36:36.889716+0000","last_undegraded":"2024-09-13T18:36:36.889716+0000","last_fullsized":"2024-09-13T18:36:36.889716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:19.625333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664807+0000","last_change":"2024-09-13T18:36:34.636621+0000","last_active":"2024-09-13T18:36:38.664807+0000","last_peered":"2024-09-13T18:36:38.664807+0000","last_clean":"2024-09-13T18:36:38.664807+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:38.664807+0000","last_undegraded":"2024-09-13T18:36:38.664807+0000","last_fullsized":"2024-09-13T18:36:38.664807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:09.196966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889523+0000","last_change":"2024-09-13T18:36:36.888348+0000","last_active":"2024-09-13T18:36:36.889523+0000","last_peered":"2024-09-13T18:36:36.889523+0000","last_clean":"2024-09-13T18:36:36.889523+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:36:36.889523+0000","last_undegraded":"2024-09-13T18:36:36.889523+0000","last_fullsized":"2024-09-13T18:36:36.889523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664780+0000","last_change":"2024-09-13T18:36:34.635305+0000","last_active":"2024-09-13T18:36:38.664780+0000","last_peered":"2024-09-13T18:36:38.664780+0000","last_clean":"2024-09-13T18:36:38.664780+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:38.664780+0000","last_undegraded":"2024-09-13T18:36:38.664780+0000","last_fullsized":"2024-09-13T18:36:38.664780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:20.849745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657275+0000","last_change":"2024-09-13T18:36:36.656517+0000","last_active":"2024-09-13T18:36:36.657275+0000","last_peered":"2024-09-13T18:36:36.657275+0000","last_clean":"2024-09-13T18:36:36.657275+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:36.657275+0000","last_undegraded":"2024-09-13T18:36:36.657275+0000","last_fullsized":"2024-09-13T18:36:36.657275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:14.465685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.663921+0000","last_change":"2024-09-13T18:36:36.656192+0000","last_active":"2024-09-13T18:36:38.663921+0000","last_peered":"2024-09-13T18:36:38.663921+0000","last_clean":"2024-09-13T18:36:38.663921+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:38.663921+0000","last_undegraded":"2024-09-13T18:36:38.663921+0000","last_fullsized":"2024-09-13T18:36:38.663921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:00.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.658738+0000","last_change":"2024-09-13T18:36:36.657279+0000","last_active":"2024-09-13T18:36:36.658738+0000","last_peered":"2024-09-13T18:36:36.658738+0000","last_clean":"2024-09-13T18:36:36.658738+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:36.658738+0000","last_undegraded":"2024-09-13T18:36:36.658738+0000","last_fullsized":"2024-09-13T18:36:36.658738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:57:41.843736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657169+0000","last_change":"2024-09-13T18:36:36.656374+0000","last_active":"2024-09-13T18:36:36.657169+0000","last_peered":"2024-09-13T18:36:36.657169+0000","last_clean":"2024-09-13T18:36:36.657169+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:36.657169+0000","last_undegraded":"2024-09-13T18:36:36.657169+0000","last_fullsized":"2024-09-13T18:36:36.657169+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:43.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664734+0000","last_change":"2024-09-13T18:36:34.636948+0000","last_active":"2024-09-13T18:36:38.664734+0000","last_peered":"2024-09-13T18:36:38.664734+0000","last_clean":"2024-09-13T18:36:38.664734+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:38.664734+0000","last_undegraded":"2024-09-13T18:36:38.664734+0000","last_fullsized":"2024-09-13T18:36:38.664734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:35.345545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051587,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":2,"up_from":28,"seq":120259084292,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":27,"seq":115964116996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":856,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":876544,"data_stored":687956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:45.254 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:36:45.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 Processor -- start 2024-09-13T18:36:45.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 -- start start 2024-09-13T18:36:45.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce58095270 0x7fce58095650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58095b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.353 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5806e080 con 0x7fce5806b1d0 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806db30 con 0x7fce58095270 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5f6c9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806e390 con 0x7fce5806cca0 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5cc64640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58095b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5d465640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce58095270 0x7fce58095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5dc66640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fce5806b1d0 0x7fce5806b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42736/0 (socket says 172.21.15.177:42736) 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.350+0000 7fce5dc66640 1 -- 172.21.15.177:0/1950192277 learned_addr learned my addr 172.21.15.177:0/1950192277 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce5d465640 1 -- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 msgr2=0x7fce58095b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce57fff640 1 -- 172.21.15.177:0/1950192277 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210725975 0 0) 0x7fce5806e080 con 0x7fce5806b1d0 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce5d465640 1 --2- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58095b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce5d465640 1 -- 172.21.15.177:0/1950192277 >> v1:172.21.15.177:6789/0 conn(0x7fce5806b1d0 legacy=0x7fce5806b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.354 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce5d465640 1 -- 172.21.15.177:0/1950192277 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5806bd90 con 0x7fce58095270 2024-09-13T18:36:45.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce5cc64640 1 --2- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58095b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:45.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce5d465640 1 --2- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce58095270 0x7fce58095650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce500097d0 tx=0x7fce5002b150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce512a17ac039569 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:45.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce57fff640 1 -- 172.21.15.177:0/1950192277 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5002bea0 con 0x7fce58095270 2024-09-13T18:36:45.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce57fff640 1 -- 172.21.15.177:0/1950192277 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50033440 con 0x7fce58095270 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.351+0000 7fce57fff640 1 -- 172.21.15.177:0/1950192277 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500337d0 con 0x7fce58095270 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 -- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce58095270 msgr2=0x7fce58095650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce58095270 0x7fce58095650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce500097d0 tx=0x7fce5002b150 comp rx=0 tx=0).stop 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 -- 172.21.15.177:0/1950192277 shutdown_connections 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58095b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/1950192277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce58095270 0x7fce58095650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 -- 172.21.15.177:0/1950192277 >> 172.21.15.177:0/1950192277 conn(0x7fce58098ba0 msgr2=0x7fce58096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 -- 172.21.15.177:0/1950192277 shutdown_connections 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.352+0000 7fce5f6c9640 1 -- 172.21.15.177:0/1950192277 wait complete. 2024-09-13T18:36:45.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 Processor -- start 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 -- start start 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 0x7fce58092f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58095270 0x7fce580939a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58121210 con 0x7fce58095270 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58121dc0 con 0x7fce5806b1d0 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5f6c9640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce581215e0 con 0x7fce5806cca0 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5d465640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 0x7fce58092f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.353+0000 7fce5d465640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 0x7fce58092f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57188/0 (socket says 172.21.15.177:57188) 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 -- 172.21.15.177:0/3247350706 learned_addr learned my addr 172.21.15.177:0/3247350706 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:45.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5dc66640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58095270 0x7fce580939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5cc64640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 -- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 msgr2=0x7fce58093460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58093460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 -- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58095270 msgr2=0x7fce580939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58095270 0x7fce580939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 -- 172.21.15.177:0/3247350706 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c009400 con 0x7fce5806b1d0 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5cc64640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58093460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5dc66640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58095270 0x7fce580939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5d465640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 0x7fce58092f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce5002b670 tx=0x7fce500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50031910 con 0x7fce5806b1d0 2024-09-13T18:36:45.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5003e070 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50031c90 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce50009400 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.354+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce58094950 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.355+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce58097ff0 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.355+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fce5004d450 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.356+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fce5007ea60 con 0x7fce5806b1d0 2024-09-13T18:36:45.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.359+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fce500600a0 con 0x7fce5806b1d0 2024-09-13T18:36:45.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.494+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce58073cc0 con 0x7fce5806b1d0 2024-09-13T18:36:45.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.495+0000 7fce55ffb640 1 -- 172.21.15.177:0/3247350706 <== mon.1 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+9086 (secure 0 0 0) 0x7fce50038070 con 0x7fce5806b1d0 2024-09-13T18:36:45.497 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:45.497 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":33,"fsid":"af843e4a-307e-4c3a-83b8-4495d5352147","created":"2024-09-13T18:34:54.195240+0000","modified":"2024-09-13T18:36:39.660415+0000","last_up_change":"2024-09-13T18:36:35.641745+0000","last_in_change":"2024-09-13T18:34:55.397217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:35:04.643396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T18:35:09.782965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b65b0a-b05a-4d13-bb43-2376cd9392da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6804","nonce":1669150124}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6806","nonce":1669150124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6810","nonce":1669150124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6809","nonce":1669150124}]},"public_addr":"172.21.15.177:6804/1669150124","cluster_addr":"172.21.15.177:6806/1669150124","heartbeat_back_addr":"172.21.15.177:6810/1669150124","heartbeat_front_addr":"172.21.15.177:6809/1669150124","state":["exists","up"]},{"osd":1,"uuid":"ab422818-9dbe-4e2f-9da7-8a78a041b09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6805","nonce":2712499331}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6807","nonce":2712499331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6811","nonce":2712499331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6808","nonce":2712499331}]},"public_addr":"172.21.15.177:6805/2712499331","cluster_addr":"172.21.15.177:6807/2712499331","heartbeat_back_addr":"172.21.15.177:6811/2712499331","heartbeat_front_addr":"172.21.15.177:6808/2712499331","state":["exists","up"]},{"osd":2,"uuid":"d0ca7945-14cb-40ee-b8a7-14586c366c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":29,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6800","nonce":341082235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6801","nonce":341082235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6803","nonce":341082235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6802","nonce":341082235}]},"public_addr":"172.21.15.177:6800/341082235","cluster_addr":"172.21.15.177:6801/341082235","heartbeat_back_addr":"172.21.15.177:6803/341082235","heartbeat_front_addr":"172.21.15.177:6802/341082235","state":["exists","up"]},{"osd":3,"uuid":"63594787-3078-431f-9ac4-e24e4bac0e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":29,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6812","nonce":2985513301}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6813","nonce":2985513301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6815","nonce":2985513301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.177:6814","nonce":2985513301}]},"public_addr":"172.21.15.177:6812/2985513301","cluster_addr":"172.21.15.177:6813/2985513301","heartbeat_back_addr":"172.21.15.177:6815/2985513301","heartbeat_front_addr":"172.21.15.177:6814/2985513301","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.556109+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.557174+0000","dead_epoch":22},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.504393+0000","dead_epoch":23},{"osd":3,"down_stamp":"2024-09-13T18:36:29.602766+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:35:00.577686+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:0/2408340411":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/2735606740":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/3625374812":"2024-09-14T18:36:22.577532+0000","172.21.15.177:0/1782828617":"2024-09-14T18:36:22.577532+0000","172.21.15.177:6816/32265":"2024-09-14T18:36:22.577532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 >> v1:172.21.15.177:6816/516212916 conn(0x7fce44047560 legacy=0x7fce44049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 msgr2=0x7fce58092f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 0x7fce58092f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce5002b670 tx=0x7fce500046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 shutdown_connections 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58095270 0x7fce580939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fce5806cca0 0x7fce58093460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 --2- 172.21.15.177:0/3247350706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5806b1d0 0x7fce58092f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 >> 172.21.15.177:0/3247350706 conn(0x7fce58098ba0 msgr2=0x7fce58099950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 shutdown_connections 2024-09-13T18:36:45.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:45.496+0000 7fce5f6c9640 1 -- 172.21.15.177:0/3247350706 wait complete. 2024-09-13T18:36:46.509 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T18:36:46.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:36:46.623 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.621+0000 7febd72b1640 1 Processor -- start 2024-09-13T18:36:46.624 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.621+0000 7febd72b1640 1 -- start start 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd72b1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 0x7febd006bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd72b1640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 0x7febd012e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd72b1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd006d050 con 0x7febd006c290 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd72b1640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd0071190 con 0x7febd012dc60 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd72b1640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febd006c670 con 0x7febd006b970 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd504d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 0x7febd006bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd584e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 0x7febd012e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd484c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7febd006c290 0x7febd012d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42748/0 (socket says 172.21.15.177:42748) 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd484c640 1 -- 172.21.15.177:0/3151700617 learned_addr learned my addr 172.21.15.177:0/3151700617 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:46.625 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd504d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 0x7febd006bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60814/0 (socket says 172.21.15.177:60814) 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd584e640 1 --2- 172.21.15.177:0/3151700617 >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 0x7febd012e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46110/0 (socket says 172.21.15.177:46110) 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd504d640 1 -- 172.21.15.177:0/3151700617 >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 msgr2=0x7febd012e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd504d640 1 --2- 172.21.15.177:0/3151700617 >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 0x7febd012e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd504d640 1 -- 172.21.15.177:0/3151700617 >> v1:172.21.15.177:6789/0 conn(0x7febd006c290 legacy=0x7febd012d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd504d640 1 -- 172.21.15.177:0/3151700617 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd006c890 con 0x7febd006b970 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febc7fff640 1 -- 172.21.15.177:0/3151700617 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922171517 0 0) 0x7febd006d050 con 0x7febd006c290 2024-09-13T18:36:46.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.622+0000 7febd584e640 1 --2- 172.21.15.177:0/3151700617 >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 0x7febd012e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.623+0000 7febd504d640 1 --2- 172.21.15.177:0/3151700617 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 0x7febd006bd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febcc009dc0 tx=0x7febcc02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ff9ea4a015d67e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.623+0000 7febc7fff640 1 -- 172.21.15.177:0/3151700617 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc007740 con 0x7febd006b970 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.623+0000 7febc7fff640 1 -- 172.21.15.177:0/3151700617 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc02d050 con 0x7febd006b970 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.623+0000 7febc7fff640 1 -- 172.21.15.177:0/3151700617 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc007ac0 con 0x7febd006b970 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 -- 172.21.15.177:0/3151700617 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 msgr2=0x7febd006bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 --2- 172.21.15.177:0/3151700617 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 0x7febd006bd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febcc009dc0 tx=0x7febcc02a560 comp rx=0 tx=0).stop 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 -- 172.21.15.177:0/3151700617 shutdown_connections 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 --2- 172.21.15.177:0/3151700617 >> v2:172.21.15.177:3300/0 conn(0x7febd012dc60 0x7febd012e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 --2- 172.21.15.177:0/3151700617 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd006b970 0x7febd006bd50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 -- 172.21.15.177:0/3151700617 >> 172.21.15.177:0/3151700617 conn(0x7febd00a01c0 msgr2=0x7febd00a05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 -- 172.21.15.177:0/3151700617 shutdown_connections 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.624+0000 7febd72b1640 1 -- 172.21.15.177:0/3151700617 wait complete. 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 Processor -- start 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 -- start start 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febd006b970 0x7febd012e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006c290 0x7febd012e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd012dc60 0x7febd012eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd0135070 con 0x7febd006c290 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd0130080 con 0x7febd006b970 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd72b1640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febd01312e0 con 0x7febd012dc60 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd484c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006c290 0x7febd012e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd584e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd012dc60 0x7febd012eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd584e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd012dc60 0x7febd012eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60816/0 (socket says 172.21.15.177:60816) 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd584e640 1 -- 172.21.15.177:0/1415963480 learned_addr learned my addr 172.21.15.177:0/1415963480 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.625+0000 7febd504d640 1 --2- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febd006b970 0x7febd012e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd484c640 1 -- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd012dc60 msgr2=0x7febd012eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd484c640 1 --2- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7febd012dc60 0x7febd012eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd484c640 1 -- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febd006b970 msgr2=0x7febd012e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd484c640 1 --2- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febd006b970 0x7febd012e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd484c640 1 -- 172.21.15.177:0/1415963480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc8009400 con 0x7febd006c290 2024-09-13T18:36:46.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd484c640 1 --2- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006c290 0x7febd012e5b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7febc000c410 tx=0x7febc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:46.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00074e0 con 0x7febd006c290 2024-09-13T18:36:46.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc0007660 con 0x7febd006c290 2024-09-13T18:36:46.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc009a70 con 0x7febd006c290 2024-09-13T18:36:46.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0004a20 con 0x7febd006c290 2024-09-13T18:36:46.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.626+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd0130730 con 0x7febd006c290 2024-09-13T18:36:46.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.627+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7febd012d780 con 0x7febd006c290 2024-09-13T18:36:46.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.627+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7febc0004b80 con 0x7febd006c290 2024-09-13T18:36:46.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.628+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7febc005c690 con 0x7febd006c290 2024-09-13T18:36:46.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.628+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 --> v1:172.21.15.177:6804/1669150124 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febb4051aa0 con 0x7febb404ddf0 2024-09-13T18:36:46.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.628+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc005ec70 con 0x7febd006c290 2024-09-13T18:36:46.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.629+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== osd.0 v1:172.21.15.177:6804/1669150124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7febb4051aa0 con 0x7febb404ddf0 2024-09-13T18:36:46.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 --> v1:172.21.15.177:6804/1669150124 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7febd0126570 con 0x7febb404ddf0 2024-09-13T18:36:46.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febc5ffb640 1 -- 172.21.15.177:0/1415963480 <== osd.0 v1:172.21.15.177:6804/1669150124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7febd0126570 con 0x7febb404ddf0 2024-09-13T18:36:46.643 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-13T18:36:46.643 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T18:36:46.643 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 >> v1:172.21.15.177:6804/1669150124 conn(0x7febb404ddf0 legacy=0x7febb4050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 >> v1:172.21.15.177:6816/516212916 conn(0x7febb4047560 legacy=0x7febb4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006c290 msgr2=0x7febd012e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 --2- 172.21.15.177:0/1415963480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006c290 0x7febd012e5b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7febc000c410 tx=0x7febc000c8d0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd584e640 1 -- 172.21.15.177:0/1415963480 reap_dead start 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 shutdown_connections 2024-09-13T18:36:46.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 >> 172.21.15.177:0/1415963480 conn(0x7febd00a01c0 msgr2=0x7febd0125680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:46.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 shutdown_connections 2024-09-13T18:36:46.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.641+0000 7febd72b1640 1 -- 172.21.15.177:0/1415963480 wait complete. 2024-09-13T18:36:46.654 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T18:36:46.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.750+0000 7f6f1cf1f640 1 Processor -- start 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.750+0000 7f6f1cf1f640 1 -- start start 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1cf1f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f1812da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1cf1f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f1812a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1cf1f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1806e4c0 con 0x7f6f18114350 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1cf1f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f18113dd0 con 0x7f6f1812df90 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1cf1f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f18114a50 con 0x7f6f1812d670 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f1812a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f15d9b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f6f18114350 0x7f6f1812a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42754/0 (socket says 172.21.15.177:42754) 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f1812da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f15d9b640 1 -- 172.21.15.177:0/1868890760 learned_addr learned my addr 172.21.15.177:0/1868890760 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f16d9d640 1 -- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 msgr2=0x7f6f1812da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f16d9d640 1 --2- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f1812da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1559a640 1 -- 172.21.15.177:0/1868890760 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274168922 0 0) 0x7f6f1806e4c0 con 0x7f6f18114350 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f16d9d640 1 -- 172.21.15.177:0/1868890760 >> v1:172.21.15.177:6789/0 conn(0x7f6f18114350 legacy=0x7f6f1812a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f16d9d640 1 -- 172.21.15.177:0/1868890760 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1806a140 con 0x7f6f1812df90 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.751+0000 7f6f1659c640 1 --2- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f1812da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:46.757 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.752+0000 7f6f16d9d640 1 --2- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f1812a8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009dc0 tx=0x7f6f1002a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257ea86ca6ccde9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.752+0000 7f6f1559a640 1 -- 172.21.15.177:0/1868890760 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10007740 con 0x7f6f1812df90 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.752+0000 7f6f1559a640 1 -- 172.21.15.177:0/1868890760 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1002d050 con 0x7f6f1812df90 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.752+0000 7f6f1559a640 1 -- 172.21.15.177:0/1868890760 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10007ac0 con 0x7f6f1812df90 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 msgr2=0x7f6f1812a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f1812a8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009dc0 tx=0x7f6f1002a510 comp rx=0 tx=0).stop 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/1868890760 shutdown_connections 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f1812a8f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/1868890760 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f1812da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/1868890760 >> 172.21.15.177:0/1868890760 conn(0x7f6f1809b680 msgr2=0x7f6f1809ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/1868890760 shutdown_connections 2024-09-13T18:36:46.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.753+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/1868890760 wait complete. 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 Processor -- start 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 -- start start 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 0x7f6f1812a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f18119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f181195a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1811b190 con 0x7f6f18114350 2024-09-13T18:36:46.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1811a720 con 0x7f6f1812df90 2024-09-13T18:36:46.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1cf1f640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f18070fb0 con 0x7f6f1812d670 2024-09-13T18:36:46.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 0x7f6f1812a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f18119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f181195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:46.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 0x7f6f1812a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46132/0 (socket says 172.21.15.177:46132) 2024-09-13T18:36:46.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.754+0000 7f6f1659c640 1 -- 172.21.15.177:0/512863782 learned_addr learned my addr 172.21.15.177:0/512863782 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f16d9d640 1 -- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 msgr2=0x7f6f18119060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f16d9d640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f18119060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f16d9d640 1 -- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 msgr2=0x7f6f1812a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f16d9d640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 0x7f6f1812a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f16d9d640 1 -- 172.21.15.177:0/512863782 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00009400 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f1659c640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 0x7f6f1812a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f16d9d640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f181195a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f10031000 tx=0x7f6f10004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10031bc0 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10031d20 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f10009a70 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10003b90 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.755+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f18134a60 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.756+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f6f10003cf0 con 0x7f6f1812df90 2024-09-13T18:36:46.763 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.756+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1812da50 con 0x7f6f1812df90 2024-09-13T18:36:46.764 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.756+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f6f1007de70 con 0x7f6f1812df90 2024-09-13T18:36:46.764 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.760+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f6f1005f510 con 0x7f6f1812df90 2024-09-13T18:36:46.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.894+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f18135bc0 con 0x7f6ef8047560 2024-09-13T18:36:46.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.895+0000 7f6f0effd640 1 -- 172.21.15.177:0/512863782 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7f6f00009c70 con 0x7f6ef8047560 2024-09-13T18:36:46.897 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 >> v1:172.21.15.177:6816/516212916 conn(0x7f6ef8047560 legacy=0x7f6ef8049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 msgr2=0x7f6f181195a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f181195a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f10031000 tx=0x7f6f10004a40 comp rx=0 tx=0).stop 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 shutdown_connections 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f1812df90 0x7f6f181195a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f6f1812d670 0x7f6f18119060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 --2- 172.21.15.177:0/512863782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f18114350 0x7f6f1812a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 >> 172.21.15.177:0/512863782 conn(0x7f6f1809b680 msgr2=0x7f6f1806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 shutdown_connections 2024-09-13T18:36:46.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:46.896+0000 7f6f1cf1f640 1 -- 172.21.15.177:0/512863782 wait complete. 2024-09-13T18:36:46.909 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T18:36:46.910 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:36:47.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.007+0000 7f2b978b1640 1 Processor -- start 2024-09-13T18:36:47.010 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.007+0000 7f2b978b1640 1 -- start start 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.007+0000 7f2b978b1640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 0x7f2b901272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b978b1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b9006e2f0 0x7f2b9006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b978b1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 0x7f2b9006c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b978b1640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9006ef80 con 0x7f2b90126ef0 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b978b1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90126d10 con 0x7f2b9006e2f0 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b978b1640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90097d20 con 0x7f2b90127810 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b9564d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 0x7f2b901272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 0x7f2b9006c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b9564d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 0x7f2b901272d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46148/0 (socket says 172.21.15.177:46148) 2024-09-13T18:36:47.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 0x7f2b9006c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60838/0 (socket says 172.21.15.177:60838) 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b9564d640 1 -- 172.21.15.177:0/657562486 learned_addr learned my addr 172.21.15.177:0/657562486 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b94e4c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b9006e2f0 0x7f2b9006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 -- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b9006e2f0 msgr2=0x7f2b9006c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 --2- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b9006e2f0 0x7f2b9006c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 -- 172.21.15.177:0/657562486 >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 msgr2=0x7f2b901272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 --2- 172.21.15.177:0/657562486 >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 0x7f2b901272d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b95e4e640 1 -- 172.21.15.177:0/657562486 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9006ead0 con 0x7f2b90127810 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b9564d640 1 --2- 172.21.15.177:0/657562486 >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 0x7f2b901272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.012 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.008+0000 7f2b94e4c640 1 --2- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b9006e2f0 0x7f2b9006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b95e4e640 1 --2- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 0x7f2b9006c820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009d90 tx=0x7f2b8002a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c495e2310615cf4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b8ffff640 1 -- 172.21.15.177:0/657562486 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80007740 con 0x7f2b90127810 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b8ffff640 1 -- 172.21.15.177:0/657562486 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8002d050 con 0x7f2b90127810 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b8ffff640 1 -- 172.21.15.177:0/657562486 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80007ac0 con 0x7f2b90127810 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 -- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 msgr2=0x7f2b9006c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 --2- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 0x7f2b9006c820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009d90 tx=0x7f2b8002a490 comp rx=0 tx=0).stop 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 -- 172.21.15.177:0/657562486 shutdown_connections 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 --2- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90127810 0x7f2b9006c820 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 --2- 172.21.15.177:0/657562486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b9006e2f0 0x7f2b9006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 --2- 172.21.15.177:0/657562486 >> v2:172.21.15.177:3300/0 conn(0x7f2b90126ef0 0x7f2b901272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.009+0000 7f2b978b1640 1 -- 172.21.15.177:0/657562486 >> 172.21.15.177:0/657562486 conn(0x7f2b90098c80 msgr2=0x7f2b90096390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.010+0000 7f2b978b1640 1 -- 172.21.15.177:0/657562486 shutdown_connections 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.010+0000 7f2b978b1640 1 -- 172.21.15.177:0/657562486 wait complete. 2024-09-13T18:36:47.018 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.010+0000 7f2b978b1640 1 Processor -- start 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.010+0000 7f2b978b1640 1 -- start start 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b9006e2f0 0x7f2b9008cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90126ef0 0x7f2b9008d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90127810 0x7f2b9008d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900903d0 con 0x7f2b9006e2f0 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9008f600 con 0x7f2b90127810 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9008f200 con 0x7f2b90126ef0 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b95e4e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90127810 0x7f2b9008d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b9006e2f0 0x7f2b9008cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b94e4c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90126ef0 0x7f2b9008d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b95e4e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90127810 0x7f2b9008d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57232/0 (socket says 172.21.15.177:57232) 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b95e4e640 1 -- 172.21.15.177:0/2881475743 learned_addr learned my addr 172.21.15.177:0/2881475743 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.019 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 -- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90126ef0 msgr2=0x7f2b9008d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 --2- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90126ef0 0x7f2b9008d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 -- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90127810 msgr2=0x7f2b9008d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 --2- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90127810 0x7f2b9008d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 -- 172.21.15.177:0/2881475743 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84009400 con 0x7f2b9006e2f0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b94e4c640 1 --2- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f2b90126ef0 0x7f2b9008d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b9564d640 1 --2- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b9006e2f0 0x7f2b9008cef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b88010dc0 tx=0x7f2b8800e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88004230 con 0x7f2b9006e2f0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b88004390 con 0x7f2b9006e2f0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88011880 con 0x7f2b9006e2f0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.011+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8800b770 con 0x7f2b9006e2f0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.012+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b80009a70 con 0x7f2b9006e2f0 2024-09-13T18:36:47.020 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.012+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b9006b1d0 con 0x7f2b9006e2f0 2024-09-13T18:36:47.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.013+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f2b880119e0 con 0x7f2b9006e2f0 2024-09-13T18:36:47.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.013+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f2b8805f5e0 con 0x7f2b9006e2f0 2024-09-13T18:36:47.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.013+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 --> v1:172.21.15.177:6805/2712499331 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b7c051aa0 con 0x7f2b7c04ddf0 2024-09-13T18:36:47.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.013+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b88061ca0 con 0x7f2b9006e2f0 2024-09-13T18:36:47.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.014+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== osd.1 v1:172.21.15.177:6805/2712499331 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f2b7c051aa0 con 0x7f2b7c04ddf0 2024-09-13T18:36:47.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 --> v1:172.21.15.177:6805/2712499331 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2b901272d0 con 0x7f2b7c04ddf0 2024-09-13T18:36:47.028 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b8dffb640 1 -- 172.21.15.177:0/2881475743 <== osd.1 v1:172.21.15.177:6805/2712499331 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f2b901272d0 con 0x7f2b7c04ddf0 2024-09-13T18:36:47.028 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-13T18:36:47.028 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T18:36:47.028 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-13T18:36:47.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 >> v1:172.21.15.177:6805/2712499331 conn(0x7f2b7c04ddf0 legacy=0x7f2b7c050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 >> v1:172.21.15.177:6816/516212916 conn(0x7f2b7c047560 legacy=0x7f2b7c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b9006e2f0 msgr2=0x7f2b9008cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 --2- 172.21.15.177:0/2881475743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b9006e2f0 0x7f2b9008cef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b88010dc0 tx=0x7f2b8800e730 comp rx=0 tx=0).stop 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b95e4e640 1 -- 172.21.15.177:0/2881475743 reap_dead start 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 shutdown_connections 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 >> 172.21.15.177:0/2881475743 conn(0x7f2b90098c80 msgr2=0x7f2b90099890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 shutdown_connections 2024-09-13T18:36:47.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.026+0000 7f2b978b1640 1 -- 172.21.15.177:0/2881475743 wait complete. 2024-09-13T18:36:47.039 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 Processor -- start 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 -- start start 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c412b730 0x7f84c412bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84c406b8e0 0x7f84c41335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c4097e00 con 0x7f84c406f230 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c406e9c0 con 0x7f84c406b8e0 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84cb165640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84c412b2e0 con 0x7f84c412b730 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c3fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f84c406f230 0x7f84c406b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42768/0 (socket says 172.21.15.177:42768) 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c8f01640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c412b730 0x7f84c412bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c3fff640 1 -- 172.21.15.177:0/3798604642 learned_addr learned my addr 172.21.15.177:0/3798604642 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c9702640 1 --2- 172.21.15.177:0/3798604642 >> v2:172.21.15.177:3300/0 conn(0x7f84c406b8e0 0x7f84c41335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c37fe640 1 -- 172.21.15.177:0/3798604642 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818500729 0 0) 0x7f84c4097e00 con 0x7f84c406f230 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c37fe640 1 -- 172.21.15.177:0/3798604642 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b0003520 con 0x7f84c406f230 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c8f01640 1 -- 172.21.15.177:0/3798604642 >> v2:172.21.15.177:3300/0 conn(0x7f84c406b8e0 msgr2=0x7f84c41335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c8f01640 1 --2- 172.21.15.177:0/3798604642 >> v2:172.21.15.177:3300/0 conn(0x7f84c406b8e0 0x7f84c41335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c8f01640 1 -- 172.21.15.177:0/3798604642 >> v1:172.21.15.177:6789/0 conn(0x7f84c406f230 legacy=0x7f84c406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c8f01640 1 -- 172.21.15.177:0/3798604642 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c406f9a0 con 0x7f84c412b730 2024-09-13T18:36:47.139 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.135+0000 7f84c9702640 1 --2- 172.21.15.177:0/3798604642 >> v2:172.21.15.177:3300/0 conn(0x7f84c406b8e0 0x7f84c41335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.136+0000 7f84c8f01640 1 --2- 172.21.15.177:0/3798604642 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c412b730 0x7f84c412bb10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097a0 tx=0x7f84bc02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2155c0f68488903f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.136+0000 7f84c37fe640 1 -- 172.21.15.177:0/3798604642 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc02adb0 con 0x7f84c412b730 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.136+0000 7f84c37fe640 1 -- 172.21.15.177:0/3798604642 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc032440 con 0x7f84c412b730 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.136+0000 7f84c37fe640 1 -- 172.21.15.177:0/3798604642 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0327d0 con 0x7f84c412b730 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 -- 172.21.15.177:0/3798604642 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c412b730 msgr2=0x7f84c412bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 --2- 172.21.15.177:0/3798604642 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c412b730 0x7f84c412bb10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097a0 tx=0x7f84bc02a060 comp rx=0 tx=0).stop 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 -- 172.21.15.177:0/3798604642 shutdown_connections 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 --2- 172.21.15.177:0/3798604642 >> v2:172.21.15.177:3300/0 conn(0x7f84c406b8e0 0x7f84c41335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 --2- 172.21.15.177:0/3798604642 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c412b730 0x7f84c412bb10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 -- 172.21.15.177:0/3798604642 >> 172.21.15.177:0/3798604642 conn(0x7f84c4098ba0 msgr2=0x7f84c4096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 -- 172.21.15.177:0/3798604642 shutdown_connections 2024-09-13T18:36:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.137+0000 7f84cb165640 1 -- 172.21.15.177:0/3798604642 wait complete. 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 Processor -- start 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 -- start start 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 0x7f84c406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84c406f230 0x7f84c406c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84c412b730 0x7f84c406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4132710 con 0x7f84c406f230 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c406dd00 con 0x7f84c412b730 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84cb165640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4125390 con 0x7f84c406b8e0 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84c8f01640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 0x7f84c406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84c8f01640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 0x7f84c406c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60868/0 (socket says 172.21.15.177:60868) 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84c8f01640 1 -- 172.21.15.177:0/2310432603 learned_addr learned my addr 172.21.15.177:0/2310432603 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.144 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84c3fff640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84c406f230 0x7f84c406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.138+0000 7f84c9702640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84c412b730 0x7f84c406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c8f01640 1 -- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84c412b730 msgr2=0x7f84c406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c8f01640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84c412b730 0x7f84c406cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c8f01640 1 -- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84c406f230 msgr2=0x7f84c406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c8f01640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84c406f230 0x7f84c406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c8f01640 1 -- 172.21.15.177:0/2310432603 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac009400 con 0x7f84c406b8e0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c3fff640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84c406f230 0x7f84c406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c8f01640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 0x7f84c406c2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84bc000c00 tx=0x7f84bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc03d850 con 0x7f84c406b8e0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc03d9d0 con 0x7f84c406b8e0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc009400 con 0x7f84c406b8e0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc03dd80 con 0x7f84c406b8e0 2024-09-13T18:36:47.145 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.139+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c406c050 con 0x7f84c406b8e0 2024-09-13T18:36:47.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.140+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c412bb10 con 0x7f84c406b8e0 2024-09-13T18:36:47.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.140+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f84bc039040 con 0x7f84c406b8e0 2024-09-13T18:36:47.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.140+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f84bc07d060 con 0x7f84c406b8e0 2024-09-13T18:36:47.146 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.144+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f84bc047cc0 con 0x7f84c406b8e0 2024-09-13T18:36:47.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.280+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f84c4073cc0 con 0x7f84a4047560 2024-09-13T18:36:47.283 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.280+0000 7f84c17fa640 1 -- 172.21.15.177:0/2310432603 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7f84c4073cc0 con 0x7f84a4047560 2024-09-13T18:36:47.283 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 >> v1:172.21.15.177:6816/516212916 conn(0x7f84a4047560 legacy=0x7f84a4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 msgr2=0x7f84c406c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 0x7f84c406c2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84bc000c00 tx=0x7f84bc0046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 shutdown_connections 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84c412b730 0x7f84c406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84c406f230 0x7f84c406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 --2- 172.21.15.177:0/2310432603 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f84c406b8e0 0x7f84c406c2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.282+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 >> 172.21.15.177:0/2310432603 conn(0x7f84c4098ba0 msgr2=0x7f84c4127da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.283+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 shutdown_connections 2024-09-13T18:36:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.283+0000 7f84cb165640 1 -- 172.21.15.177:0/2310432603 wait complete. 2024-09-13T18:36:47.296 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T18:36:47.296 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:36:47.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 Processor -- start 2024-09-13T18:36:47.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 -- start start 2024-09-13T18:36:47.397 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3d1006e2f0 0x7f3d1012f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 0x7f3d10131d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d10126d10 con 0x7f3d10126ef0 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1006ef80 con 0x7f3d1006e2f0 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.394+0000 7f3d1559c640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10097d20 con 0x7f3d1012f990 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 0x7f3d10131d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0e7fc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3d1006e2f0 0x7f3d1012f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0effd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f3d10126ef0 0x7f3d101272d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42774/0 (socket says 172.21.15.177:42774) 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0f7fe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 0x7f3d10131d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60882/0 (socket says 172.21.15.177:60882) 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0effd640 1 -- 172.21.15.177:0/4223419407 learned_addr learned my addr 172.21.15.177:0/4223419407 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:47.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0dffb640 1 -- 172.21.15.177:0/4223419407 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98379551 0 0) 0x7f3d10126d10 con 0x7f3d10126ef0 2024-09-13T18:36:47.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0f7fe640 1 -- 172.21.15.177:0/4223419407 >> v2:172.21.15.177:3300/0 conn(0x7f3d1006e2f0 msgr2=0x7f3d1012f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0f7fe640 1 --2- 172.21.15.177:0/4223419407 >> v2:172.21.15.177:3300/0 conn(0x7f3d1006e2f0 0x7f3d1012f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0f7fe640 1 -- 172.21.15.177:0/4223419407 >> v1:172.21.15.177:6789/0 conn(0x7f3d10126ef0 legacy=0x7f3d101272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.399 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.395+0000 7f3d0f7fe640 1 -- 172.21.15.177:0/4223419407 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d101279e0 con 0x7f3d1012f990 2024-09-13T18:36:47.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d0f7fe640 1 --2- 172.21.15.177:0/4223419407 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 0x7f3d10131d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009d90 tx=0x7f3d0802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840f45cb7d23a254 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d0dffb640 1 -- 172.21.15.177:0/4223419407 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08031600 con 0x7f3d1012f990 2024-09-13T18:36:47.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d0dffb640 1 -- 172.21.15.177:0/4223419407 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0802d050 con 0x7f3d1012f990 2024-09-13T18:36:47.402 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d0dffb640 1 -- 172.21.15.177:0/4223419407 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08031980 con 0x7f3d1012f990 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 -- 172.21.15.177:0/4223419407 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 msgr2=0x7f3d10131d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 --2- 172.21.15.177:0/4223419407 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 0x7f3d10131d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009d90 tx=0x7f3d0802a5d0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 -- 172.21.15.177:0/4223419407 shutdown_connections 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 --2- 172.21.15.177:0/4223419407 >> v2:172.21.15.177:3300/0 conn(0x7f3d1006e2f0 0x7f3d1012f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 --2- 172.21.15.177:0/4223419407 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1012f990 0x7f3d10131d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 -- 172.21.15.177:0/4223419407 >> 172.21.15.177:0/4223419407 conn(0x7f3d10098c80 msgr2=0x7f3d10096390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 -- 172.21.15.177:0/4223419407 shutdown_connections 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.396+0000 7f3d1559c640 1 -- 172.21.15.177:0/4223419407 wait complete. 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.397+0000 7f3d1559c640 1 Processor -- start 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.397+0000 7f3d1559c640 1 -- start start 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1006e2f0 0x7f3d1008d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d10126ef0 0x7f3d1008d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d1012f990 0x7f3d1008dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1008b300 con 0x7f3d10126ef0 2024-09-13T18:36:47.403 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1008cf80 con 0x7f3d1012f990 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100715c0 con 0x7f3d1006e2f0 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1006e2f0 0x7f3d1008d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0e7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d10126ef0 0x7f3d1008d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1006e2f0 0x7f3d1008d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60894/0 (socket says 172.21.15.177:60894) 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 -- 172.21.15.177:0/1077508330 learned_addr learned my addr 172.21.15.177:0/1077508330 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0f7fe640 1 --2- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d1012f990 0x7f3d1008dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 -- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d1012f990 msgr2=0x7f3d1008dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 --2- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d1012f990 0x7f3d1008dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 -- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d10126ef0 msgr2=0x7f3d1008d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 --2- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d10126ef0 0x7f3d1008d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 -- 172.21.15.177:0/1077508330 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08009a70 con 0x7f3d1006e2f0 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0e7fc640 1 --2- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d10126ef0 0x7f3d1008d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d0effd640 1 --2- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1006e2f0 0x7f3d1008d1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000e760 tx=0x7f3d0000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000ee70 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d000072e0 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1008ea40 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.398+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d000076b0 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.399+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1008c520 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.399+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d101272d0 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.399+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3d0000c850 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.400+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f3d0005ec50 con 0x7f3d1006e2f0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.400+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 --> v1:172.21.15.177:6800/341082235 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cfc051aa0 con 0x7f3cfc04ddf0 2024-09-13T18:36:47.405 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.400+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d00061310 con 0x7f3d1006e2f0 2024-09-13T18:36:47.406 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.401+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== osd.2 v1:172.21.15.177:6800/341082235 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f3cfc051aa0 con 0x7f3cfc04ddf0 2024-09-13T18:36:47.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.412+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 --> v1:172.21.15.177:6800/341082235 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3d10132500 con 0x7f3cfc04ddf0 2024-09-13T18:36:47.415 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3cf77fe640 1 -- 172.21.15.177:0/1077508330 <== osd.2 v1:172.21.15.177:6800/341082235 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f3d10132500 con 0x7f3cfc04ddf0 2024-09-13T18:36:47.415 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-13T18:36:47.415 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T18:36:47.415 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-13T18:36:47.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 >> v1:172.21.15.177:6800/341082235 conn(0x7f3cfc04ddf0 legacy=0x7f3cfc050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 >> v1:172.21.15.177:6816/516212916 conn(0x7f3cfc047560 legacy=0x7f3cfc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1006e2f0 msgr2=0x7f3d1008d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 --2- 172.21.15.177:0/1077508330 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f3d1006e2f0 0x7f3d1008d1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000e760 tx=0x7f3d0000ec20 comp rx=0 tx=0).stop 2024-09-13T18:36:47.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d0f7fe640 1 -- 172.21.15.177:0/1077508330 reap_dead start 2024-09-13T18:36:47.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 shutdown_connections 2024-09-13T18:36:47.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 >> 172.21.15.177:0/1077508330 conn(0x7f3d10098c80 msgr2=0x7f3d1006f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 shutdown_connections 2024-09-13T18:36:47.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.413+0000 7f3d1559c640 1 -- 172.21.15.177:0/1077508330 wait complete. 2024-09-13T18:36:47.425 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T18:36:47.524 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 Processor -- start 2024-09-13T18:36:47.524 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 -- start start 2024-09-13T18:36:47.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa7e812e6c0 0x7fa7e8130ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e81333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e8098130 con 0x7fa7e812b770 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e809a190 con 0x7fa7e812e6c0 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ef861640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8096570 con 0x7fa7e8130ff0 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7eddfe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e81333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ecdfc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa7e812e6c0 0x7fa7e8130ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ed5fd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa7e812b770 0x7fa7e812bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42776/0 (socket says 172.21.15.177:42776) 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.522+0000 7fa7ed5fd640 1 -- 172.21.15.177:0/2930380104 learned_addr learned my addr 172.21.15.177:0/2930380104 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7eddfe640 1 -- 172.21.15.177:0/2930380104 >> v2:172.21.15.177:3300/0 conn(0x7fa7e812e6c0 msgr2=0x7fa7e8130ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7eddfe640 1 --2- 172.21.15.177:0/2930380104 >> v2:172.21.15.177:3300/0 conn(0x7fa7e812e6c0 0x7fa7e8130ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7eddfe640 1 -- 172.21.15.177:0/2930380104 >> v1:172.21.15.177:6789/0 conn(0x7fa7e812b770 legacy=0x7fa7e812bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7eddfe640 1 -- 172.21.15.177:0/2930380104 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e812b3b0 con 0x7fa7e8130ff0 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7e7fff640 1 -- 172.21.15.177:0/2930380104 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923787333 0 0) 0x7fa7e8098130 con 0x7fa7e812b770 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7ecdfc640 1 --2- 172.21.15.177:0/2930380104 >> v2:172.21.15.177:3300/0 conn(0x7fa7e812e6c0 0x7fa7e8130ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7eddfe640 1 --2- 172.21.15.177:0/2930380104 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e81333e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80097d0 tx=0x7fa7d802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f40de20fa86c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.523+0000 7fa7e7fff640 1 -- 172.21.15.177:0/2930380104 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d802ae40 con 0x7fa7e8130ff0 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7e7fff640 1 -- 172.21.15.177:0/2930380104 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8032440 con 0x7fa7e8130ff0 2024-09-13T18:36:47.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7e7fff640 1 -- 172.21.15.177:0/2930380104 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80327d0 con 0x7fa7e8130ff0 2024-09-13T18:36:47.530 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 -- 172.21.15.177:0/2930380104 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 msgr2=0x7fa7e81333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.530 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 --2- 172.21.15.177:0/2930380104 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e81333e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80097d0 tx=0x7fa7d802a130 comp rx=0 tx=0).stop 2024-09-13T18:36:47.530 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 -- 172.21.15.177:0/2930380104 shutdown_connections 2024-09-13T18:36:47.530 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 --2- 172.21.15.177:0/2930380104 >> v2:172.21.15.177:3300/0 conn(0x7fa7e812e6c0 0x7fa7e8130ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.530 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 --2- 172.21.15.177:0/2930380104 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e81333e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 -- 172.21.15.177:0/2930380104 >> 172.21.15.177:0/2930380104 conn(0x7fa7e809ab20 msgr2=0x7fa7e809af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 -- 172.21.15.177:0/2930380104 shutdown_connections 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.524+0000 7fa7ef861640 1 -- 172.21.15.177:0/2930380104 wait complete. 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 Processor -- start 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 -- start start 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 0x7fa7e80705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7e812e6c0 0x7fa7e8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e8071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e806fcd0 con 0x7fa7e812b770 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e806f4f0 con 0x7fa7e812e6c0 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.525+0000 7fa7ef861640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e806ec80 con 0x7fa7e8130ff0 2024-09-13T18:36:47.531 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 0x7fa7e80705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7eddfe640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e8071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 0x7fa7e80705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46204/0 (socket says 172.21.15.177:46204) 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 -- 172.21.15.177:0/459802220 learned_addr learned my addr 172.21.15.177:0/459802220 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ecdfc640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7e812e6c0 0x7fa7e8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 -- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 msgr2=0x7fa7e8071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e8071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 -- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7e812e6c0 msgr2=0x7fa7e8070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7e812e6c0 0x7fa7e8070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 -- 172.21.15.177:0/459802220 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc009a70 con 0x7fa7e812b770 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ecdfc640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7e812e6c0 0x7fa7e8070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ed5fd640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 0x7fa7e80705d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c3f0 tx=0x7fa7e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.532 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00074e0 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e0007640 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d8009400 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0004a00 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.526+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e8096e60 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.527+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa7e0004b60 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.527+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e812bb50 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.528+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fa7e005c620 con 0x7fa7e812b770 2024-09-13T18:36:47.533 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.531+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fa7e003a0c0 con 0x7fa7e812b770 2024-09-13T18:36:47.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.665+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e8099e90 con 0x7fa7d4047560 2024-09-13T18:36:47.668 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.665+0000 7fa7e5ffb640 1 -- 172.21.15.177:0/459802220 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7fa7dc00f040 con 0x7fa7d4047560 2024-09-13T18:36:47.668 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T18:36:47.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 >> v1:172.21.15.177:6816/516212916 conn(0x7fa7d4047560 legacy=0x7fa7d4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 msgr2=0x7fa7e80705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 0x7fa7e80705d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c3f0 tx=0x7fa7e000c8b0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 shutdown_connections 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7fa7e8130ff0 0x7fa7e8071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7e812e6c0 0x7fa7e8070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 --2- 172.21.15.177:0/459802220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7e812b770 0x7fa7e80705d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 >> 172.21.15.177:0/459802220 conn(0x7fa7e809ab20 msgr2=0x7fa7e80979f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 shutdown_connections 2024-09-13T18:36:47.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.667+0000 7fa7ef861640 1 -- 172.21.15.177:0/459802220 wait complete. 2024-09-13T18:36:47.680 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T18:36:47.680 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:36:47.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 Processor -- start 2024-09-13T18:36:47.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 -- start start 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec12ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06bcc0 0x7f5aec133560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec06ff70 con 0x7f5aec06b1d0 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06eb50 con 0x7f5aec12b6b0 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.778+0000 7f5af1219640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec097d20 con 0x7f5aec06bcc0 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aea59c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5aec06b1d0 0x7f5aec06b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42780/0 (socket says 172.21.15.177:42780) 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aea59c640 1 -- 172.21.15.177:0/2712166148 learned_addr learned my addr 172.21.15.177:0/2712166148 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aeb59e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06bcc0 0x7f5aec133560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aead9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec12ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5ae9d9b640 1 -- 172.21.15.177:0/2712166148 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619302431 0 0) 0x7f5aec06ff70 con 0x7f5aec06b1d0 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aead9d640 1 -- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06bcc0 msgr2=0x7f5aec133560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aead9d640 1 --2- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06bcc0 0x7f5aec133560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aead9d640 1 -- 172.21.15.177:0/2712166148 >> v1:172.21.15.177:6789/0 conn(0x7f5aec06b1d0 legacy=0x7f5aec06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aead9d640 1 -- 172.21.15.177:0/2712166148 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec12b260 con 0x7f5aec12b6b0 2024-09-13T18:36:47.783 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.779+0000 7f5aeb59e640 1 --2- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06bcc0 0x7f5aec133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5aead9d640 1 --2- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec12ba90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097d0 tx=0x7f5adc02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbf6f53db4f7e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5ae9d9b640 1 -- 172.21.15.177:0/2712166148 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc02ad90 con 0x7f5aec12b6b0 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5ae9d9b640 1 -- 172.21.15.177:0/2712166148 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc032440 con 0x7f5aec12b6b0 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5ae9d9b640 1 -- 172.21.15.177:0/2712166148 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0327d0 con 0x7f5aec12b6b0 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 -- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 msgr2=0x7f5aec12ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 --2- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec12ba90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097d0 tx=0x7f5adc02a040 comp rx=0 tx=0).stop 2024-09-13T18:36:47.786 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 -- 172.21.15.177:0/2712166148 shutdown_connections 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 --2- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06bcc0 0x7f5aec133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 --2- 172.21.15.177:0/2712166148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec12ba90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 -- 172.21.15.177:0/2712166148 >> 172.21.15.177:0/2712166148 conn(0x7f5aec098c80 msgr2=0x7f5aec096390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 -- 172.21.15.177:0/2712166148 shutdown_connections 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.780+0000 7f5af1219640 1 -- 172.21.15.177:0/2712166148 wait complete. 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 Processor -- start 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 -- start start 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06b1d0 0x7f5aec1c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5aec06bcc0 0x7f5aec1c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec125110 con 0x7f5aec06bcc0 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec1230a0 con 0x7f5aec12b6b0 2024-09-13T18:36:47.787 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5af1219640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec124d60 con 0x7f5aec06b1d0 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06b1d0 0x7f5aec1c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aea59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5aec06bcc0 0x7f5aec1c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06b1d0 0x7f5aec1c0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:60940/0 (socket says 172.21.15.177:60940) 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 -- 172.21.15.177:0/1181006392 learned_addr learned my addr 172.21.15.177:0/1181006392 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aea59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5aec06bcc0 0x7f5aec1c11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46216/0 (socket says 172.21.15.177:46216) 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aeb59e640 1 --2- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 -- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 msgr2=0x7f5aec121640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 --2- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec121640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 -- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5aec06bcc0 msgr2=0x7f5aec1c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 --2- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5aec06bcc0 0x7f5aec1c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aead9d640 1 -- 172.21.15.177:0/1181006392 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009a70 con 0x7f5aec06b1d0 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.782+0000 7f5aeb59e640 1 --2- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5aec12b6b0 0x7f5aec121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.783+0000 7f5aead9d640 1 --2- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06b1d0 0x7f5aec1c0c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5adc02a080 tx=0x7f5adc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.783+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc03d850 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.783+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc03d9b0 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.783+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc009400 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.783+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc03dd60 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.783+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec1c16e0 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.784+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5aec070d30 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.784+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5adc039040 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.784+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f5adc07d050 con 0x7f5aec06b1d0 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.784+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 --> v1:172.21.15.177:6812/2985513301 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ad8051ad0 con 0x7f5ad804de20 2024-09-13T18:36:47.789 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.784+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5adc07f710 con 0x7f5aec06b1d0 2024-09-13T18:36:47.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.785+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== osd.3 v1:172.21.15.177:6812/2985513301 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f5ad8051ad0 con 0x7f5ad804de20 2024-09-13T18:36:47.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.797+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 --> v1:172.21.15.177:6812/2985513301 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5aec09ada0 con 0x7f5ad804de20 2024-09-13T18:36:47.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.797+0000 7f5ad37fe640 1 -- 172.21.15.177:0/1181006392 <== osd.3 v1:172.21.15.177:6812/2985513301 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f5aec09ada0 con 0x7f5ad804de20 2024-09-13T18:36:47.799 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-13T18:36:47.799 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T18:36:47.800 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-13T18:36:47.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.797+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 >> v1:172.21.15.177:6812/2985513301 conn(0x7f5ad804de20 legacy=0x7f5ad8050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.797+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 >> v1:172.21.15.177:6816/516212916 conn(0x7f5ad8047590 legacy=0x7f5ad8049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06b1d0 msgr2=0x7f5aec1c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5af1219640 1 --2- 172.21.15.177:0/1181006392 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f5aec06b1d0 0x7f5aec1c0c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5adc02a080 tx=0x7f5adc0046c0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5aeb59e640 1 -- 172.21.15.177:0/1181006392 reap_dead start 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 shutdown_connections 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 >> 172.21.15.177:0/1181006392 conn(0x7f5aec098c80 msgr2=0x7f5aec099890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 shutdown_connections 2024-09-13T18:36:47.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.798+0000 7f5af1219640 1 -- 172.21.15.177:0/1181006392 wait complete. 2024-09-13T18:36:47.810 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T18:36:47.906 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.904+0000 7f22f032c640 1 Processor -- start 2024-09-13T18:36:47.907 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.904+0000 7f22f032c640 1 -- start start 2024-09-13T18:36:47.908 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22f032c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806c640 0x7f22e806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.908 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22f032c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e806cf60 0x7f22e812a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.908 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22f032c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f22e812a700 0x7f22e812abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.908 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22f032c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e806c460 con 0x7f22e812a700 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22f032c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8097e00 con 0x7f22e806c640 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22f032c640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22e806d770 con 0x7f22e806cf60 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ed8c7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e806cf60 0x7f22e812a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f22e812a700 0x7f22e812abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee0c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806c640 0x7f22e806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f22e812a700 0x7f22e812abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46232/0 (socket says 172.21.15.177:46232) 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 -- 172.21.15.177:0/838681804 learned_addr learned my addr 172.21.15.177:0/838681804 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 -- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e806cf60 msgr2=0x7f22e812a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e806cf60 0x7f22e812a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 -- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806c640 msgr2=0x7f22e806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806c640 0x7f22e806ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.909 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee8c9640 1 -- 172.21.15.177:0/838681804 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e806e0b0 con 0x7f22e812a700 2024-09-13T18:36:47.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.905+0000 7f22ee0c8640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806c640 0x7f22e806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:47.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.906+0000 7f22ee8c9640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e812a700 0x7f22e812abb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009e50 tx=0x7f22e002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992bb8fc264f1726 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.906+0000 7f22ed0c6640 1 -- 172.21.15.177:0/838681804 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0031600 con 0x7f22e812a700 2024-09-13T18:36:47.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.906+0000 7f22ed0c6640 1 -- 172.21.15.177:0/838681804 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e002d050 con 0x7f22e812a700 2024-09-13T18:36:47.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.906+0000 7f22ed0c6640 1 -- 172.21.15.177:0/838681804 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0031980 con 0x7f22e812a700 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 -- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e812a700 msgr2=0x7f22e812abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e812a700 0x7f22e812abb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009e50 tx=0x7f22e002a560 comp rx=0 tx=0).stop 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 -- 172.21.15.177:0/838681804 shutdown_connections 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e812a700 0x7f22e812abb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e806cf60 0x7f22e812a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 --2- 172.21.15.177:0/838681804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806c640 0x7f22e806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 -- 172.21.15.177:0/838681804 >> 172.21.15.177:0/838681804 conn(0x7f22e8098ba0 msgr2=0x7f22e8096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 -- 172.21.15.177:0/838681804 shutdown_connections 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.907+0000 7f22f032c640 1 -- 172.21.15.177:0/838681804 wait complete. 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 Processor -- start 2024-09-13T18:36:47.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 -- start start 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e806c640 0x7f22e812ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806cf60 0x7f22e812bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e812a700 0x7f22e812c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e812e680 con 0x7f22e806c640 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e812ee40 con 0x7f22e806cf60 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22f032c640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22e812f150 con 0x7f22e812a700 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ee0c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e806c640 0x7f22e812ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ed8c7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806cf60 0x7f22e812bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ee0c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e806c640 0x7f22e812ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:46238/0 (socket says 172.21.15.177:46238) 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ee0c8640 1 -- 172.21.15.177:0/2474023995 learned_addr learned my addr 172.21.15.177:0/2474023995 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ee8c9640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e812a700 0x7f22e812c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ed8c7640 1 -- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e812a700 msgr2=0x7f22e812c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ed8c7640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e812a700 0x7f22e812c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ed8c7640 1 -- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e806c640 msgr2=0x7f22e812ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ed8c7640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e806c640 0x7f22e812ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.908+0000 7f22ed8c7640 1 -- 172.21.15.177:0/2474023995 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009400 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.909+0000 7f22ed8c7640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806cf60 0x7f22e812bf60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e820 tx=0x7f22dc00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.909+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00c940 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.909+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00caa0 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.909+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e0009b30 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.909+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00ce30 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.909+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e812dd30 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.910+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f22dc013470 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.910+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e812a1c0 con 0x7f22e806cf60 2024-09-13T18:36:47.918 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.910+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f22dc05ca30 con 0x7f22e806cf60 2024-09-13T18:36:47.919 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:47.914+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f22dc03a620 con 0x7f22e806cf60 2024-09-13T18:36:48.051 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.049+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f22e8073cc0 con 0x7f22cc047560 2024-09-13T18:36:48.051 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.049+0000 7f22daffd640 1 -- 172.21.15.177:0/2474023995 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7f22e8073cc0 con 0x7f22cc047560 2024-09-13T18:36:48.051 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 >> v1:172.21.15.177:6816/516212916 conn(0x7f22cc047560 legacy=0x7f22cc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806cf60 msgr2=0x7f22e812bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806cf60 0x7f22e812bf60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e820 tx=0x7f22dc00ece0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 shutdown_connections 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f22e812a700 0x7f22e812c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e806cf60 0x7f22e812bf60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.050+0000 7f22f032c640 1 --2- 172.21.15.177:0/2474023995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e806c640 0x7f22e812ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.051+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 >> 172.21.15.177:0/2474023995 conn(0x7f22e8098ba0 msgr2=0x7f22e806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:48.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.051+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 shutdown_connections 2024-09-13T18:36:48.054 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.051+0000 7f22f032c640 1 -- 172.21.15.177:0/2474023995 wait complete. 2024-09-13T18:36:48.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:36:48.161 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.158+0000 7f8335fc7640 1 Processor -- start 2024-09-13T18:36:48.161 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.158+0000 7f8335fc7640 1 -- start start 2024-09-13T18:36:48.161 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f8335fc7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f833012b730 0x7f833012bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f8335fc7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833006f230 0x7f833006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f8335fc7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8330097e00 con 0x7f833006b710 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f8335fc7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833006e9c0 con 0x7f833012b730 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f8335fc7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f833012b2e0 con 0x7f833006f230 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f833012b730 0x7f833012bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832effd640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833006f230 0x7f833006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f833006b710 0x7f83301325b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:42794/0 (socket says 172.21.15.177:42794) 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832ffff640 1 -- 172.21.15.177:0/3424526792 learned_addr learned my addr 172.21.15.177:0/3424526792 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832effd640 1 -- 172.21.15.177:0/3424526792 >> v2:172.21.15.177:3300/0 conn(0x7f833012b730 msgr2=0x7f833012bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832effd640 1 --2- 172.21.15.177:0/3424526792 >> v2:172.21.15.177:3300/0 conn(0x7f833012b730 0x7f833012bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832effd640 1 -- 172.21.15.177:0/3424526792 >> v1:172.21.15.177:6789/0 conn(0x7f833006b710 legacy=0x7f83301325b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832effd640 1 -- 172.21.15.177:0/3424526792 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833006f9a0 con 0x7f833006f230 2024-09-13T18:36:48.162 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.159+0000 7f832f7fe640 1 --2- 172.21.15.177:0/3424526792 >> v2:172.21.15.177:3300/0 conn(0x7f833012b730 0x7f833012bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:48.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f832effd640 1 --2- 172.21.15.177:0/3424526792 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833006f230 0x7f833006b1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8324009d90 tx=0x7f832402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84806a4103804d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:48.165 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f832e7fc640 1 -- 172.21.15.177:0/3424526792 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324007740 con 0x7f833006f230 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f832e7fc640 1 -- 172.21.15.177:0/3424526792 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832402d050 con 0x7f833006f230 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f832e7fc640 1 -- 172.21.15.177:0/3424526792 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324007ac0 con 0x7f833006f230 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 -- 172.21.15.177:0/3424526792 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833006f230 msgr2=0x7f833006b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3424526792 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833006f230 0x7f833006b1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8324009d90 tx=0x7f832402a530 comp rx=0 tx=0).stop 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 -- 172.21.15.177:0/3424526792 shutdown_connections 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3424526792 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833006f230 0x7f833006b1d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3424526792 >> v2:172.21.15.177:3300/0 conn(0x7f833012b730 0x7f833012bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 -- 172.21.15.177:0/3424526792 >> 172.21.15.177:0/3424526792 conn(0x7f8330098ba0 msgr2=0x7f8330096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 -- 172.21.15.177:0/3424526792 shutdown_connections 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.160+0000 7f8335fc7640 1 -- 172.21.15.177:0/3424526792 wait complete. 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.161+0000 7f8335fc7640 1 Processor -- start 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.161+0000 7f8335fc7640 1 -- start start 2024-09-13T18:36:48.166 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8335fc7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 0x7f833008af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8335fc7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f833006f230 0x7f833008b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8335fc7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833012b730 0x7f833008b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8335fc7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8330095aa0 con 0x7f833006f230 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8335fc7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833008e590 con 0x7f833006b710 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8335fc7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f833008c8d0 con 0x7f833012b730 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 0x7f833008af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 0x7f833008af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:57294/0 (socket says 172.21.15.177:57294) 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832effd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f833006f230 0x7f833008b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 -- 172.21.15.177:0/3748317346 learned_addr learned my addr 172.21.15.177:0/3748317346 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832ffff640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833012b730 0x7f833008b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 -- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833012b730 msgr2=0x7f833008b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833012b730 0x7f833008b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.167 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 -- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f833006f230 msgr2=0x7f833008b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f833006f230 0x7f833008b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 -- 172.21.15.177:0/3748317346 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320009400 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f832f7fe640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 0x7f833008af60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f832000ebe0 tx=0x7f832000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mon.1 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320009760 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.162+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mon.1 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832000cc30 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.163+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mon.1 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000eee0 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.163+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8324009a70 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.163+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83301c01a0 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.163+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833006b1d0 con 0x7f833006b710 2024-09-13T18:36:48.168 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.164+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mon.1 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f8320013470 con 0x7f833006b710 2024-09-13T18:36:48.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.164+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mon.1 v2:172.21.15.177:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f832005cbe0 con 0x7f833006b710 2024-09-13T18:36:48.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.167+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mon.1 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f832003a7f0 con 0x7f833006b710 2024-09-13T18:36:48.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.298+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f833008d2a0 con 0x7f831c047560 2024-09-13T18:36:48.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.299+0000 7f8317fff640 1 -- 172.21.15.177:0/3748317346 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34752 (unknown 2965378022 0 3303029602) 0x7f8324007540 con 0x7f831c047560 2024-09-13T18:36:48.302 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:36:48.302 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:36:48.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 >> v1:172.21.15.177:6816/516212916 conn(0x7f831c047560 legacy=0x7f831c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 msgr2=0x7f833008af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:48.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 0x7f833008af60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f832000ebe0 tx=0x7f832000c520 comp rx=0 tx=0).stop 2024-09-13T18:36:48.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 shutdown_connections 2024-09-13T18:36:48.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f833012b730 0x7f833008b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f833006f230 0x7f833008b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 --2- 172.21.15.177:0/3748317346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f833006b710 0x7f833008af60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:48.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 >> 172.21.15.177:0/3748317346 conn(0x7f8330098ba0 msgr2=0x7f8330099870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:48.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 shutdown_connections 2024-09-13T18:36:48.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:36:48.301+0000 7f8335fc7640 1 -- 172.21.15.177:0/3748317346 wait complete. 2024-09-13T18:36:48.314 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-13T18:36:46.594380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112328,"kb_used_data":2252,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375032,"statfs":{"total":386547056640,"available":386432032768,"internally_reserved":0,"allocated":2306048,"data_stored":1570439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001446"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889716+0000","last_change":"2024-09-13T18:36:36.889220+0000","last_active":"2024-09-13T18:36:36.889716+0000","last_peered":"2024-09-13T18:36:36.889716+0000","last_clean":"2024-09-13T18:36:36.889716+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:36:36.889716+0000","last_undegraded":"2024-09-13T18:36:36.889716+0000","last_fullsized":"2024-09-13T18:36:36.889716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:19.625333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664807+0000","last_change":"2024-09-13T18:36:34.636621+0000","last_active":"2024-09-13T18:36:38.664807+0000","last_peered":"2024-09-13T18:36:38.664807+0000","last_clean":"2024-09-13T18:36:38.664807+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:38.664807+0000","last_undegraded":"2024-09-13T18:36:38.664807+0000","last_fullsized":"2024-09-13T18:36:38.664807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:09.196966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.889523+0000","last_change":"2024-09-13T18:36:36.888348+0000","last_active":"2024-09-13T18:36:36.889523+0000","last_peered":"2024-09-13T18:36:36.889523+0000","last_clean":"2024-09-13T18:36:36.889523+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:36:36.889523+0000","last_undegraded":"2024-09-13T18:36:36.889523+0000","last_fullsized":"2024-09-13T18:36:36.889523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664780+0000","last_change":"2024-09-13T18:36:34.635305+0000","last_active":"2024-09-13T18:36:38.664780+0000","last_peered":"2024-09-13T18:36:38.664780+0000","last_clean":"2024-09-13T18:36:38.664780+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:38.664780+0000","last_undegraded":"2024-09-13T18:36:38.664780+0000","last_fullsized":"2024-09-13T18:36:38.664780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:20.849745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657275+0000","last_change":"2024-09-13T18:36:36.656517+0000","last_active":"2024-09-13T18:36:36.657275+0000","last_peered":"2024-09-13T18:36:36.657275+0000","last_clean":"2024-09-13T18:36:36.657275+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:36.657275+0000","last_undegraded":"2024-09-13T18:36:36.657275+0000","last_fullsized":"2024-09-13T18:36:36.657275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:14.465685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.663921+0000","last_change":"2024-09-13T18:36:36.656192+0000","last_active":"2024-09-13T18:36:38.663921+0000","last_peered":"2024-09-13T18:36:38.663921+0000","last_clean":"2024-09-13T18:36:38.663921+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:38.663921+0000","last_undegraded":"2024-09-13T18:36:38.663921+0000","last_fullsized":"2024-09-13T18:36:38.663921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:00.992927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.658738+0000","last_change":"2024-09-13T18:36:36.657279+0000","last_active":"2024-09-13T18:36:36.658738+0000","last_peered":"2024-09-13T18:36:36.658738+0000","last_clean":"2024-09-13T18:36:36.658738+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:36.658738+0000","last_undegraded":"2024-09-13T18:36:36.658738+0000","last_fullsized":"2024-09-13T18:36:36.658738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:57:41.843736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T18:36:36.657169+0000","last_change":"2024-09-13T18:36:36.656374+0000","last_active":"2024-09-13T18:36:36.657169+0000","last_peered":"2024-09-13T18:36:36.657169+0000","last_clean":"2024-09-13T18:36:36.657169+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:36.657169+0000","last_undegraded":"2024-09-13T18:36:36.657169+0000","last_fullsized":"2024-09-13T18:36:36.657169+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:10.712907+0000","last_clean_scrub_stamp":"2024-09-13T18:35:10.712907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:43.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T18:36:38.664734+0000","last_change":"2024-09-13T18:36:34.636948+0000","last_active":"2024-09-13T18:36:38.664734+0000","last_peered":"2024-09-13T18:36:38.664734+0000","last_clean":"2024-09-13T18:36:38.664734+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:38.664734+0000","last_undegraded":"2024-09-13T18:36:38.664734+0000","last_fullsized":"2024-09-13T18:36:38.664734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:35:05.623580+0000","last_clean_scrub_stamp":"2024-09-13T18:35:05.623580+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:35.345545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051588,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]}]},{"osd":2,"up_from":28,"seq":120259084292,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":27,"seq":115964116996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":856,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":876544,"data_stored":687956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:36:48.318 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.318 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T18:35:10.712907+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T18:35:05.623580+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=35, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=36, tm_sec=45, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:36:48.319 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:37:08.321 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:37:08.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 Processor -- start 2024-09-13T18:37:08.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 -- start start 2024-09-13T18:37:08.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 0x7f308806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3088095710 0x7f3088133790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308806fb40 con 0x7f308806baf0 2024-09-13T18:37:08.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3088097e00 con 0x7f3088095710 2024-09-13T18:37:08.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.427+0000 7f30902b7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f308806fd70 con 0x7f308806b1d0 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 0x7f308806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e854640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3088095710 0x7f3088133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 0x7f308806b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:39842/0 (socket says 172.21.15.177:39842) 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308d852640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f308806baf0 0x7f3088095000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47678/0 (socket says 172.21.15.177:47678) 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 -- 172.21.15.177:0/3119952797 learned_addr learned my addr 172.21.15.177:0/3119952797 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 -- 172.21.15.177:0/3119952797 >> v2:172.21.15.177:3300/0 conn(0x7f3088095710 msgr2=0x7f3088133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 --2- 172.21.15.177:0/3119952797 >> v2:172.21.15.177:3300/0 conn(0x7f3088095710 0x7f3088133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 -- 172.21.15.177:0/3119952797 >> v1:172.21.15.177:6789/0 conn(0x7f308806baf0 legacy=0x7f3088095000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e053640 1 -- 172.21.15.177:0/3119952797 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308806f1c0 con 0x7f308806b1d0 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308d051640 1 -- 172.21.15.177:0/3119952797 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425861835 0 0) 0x7f308806fb40 con 0x7f308806baf0 2024-09-13T18:37:08.432 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.428+0000 7f308e854640 1 --2- 172.21.15.177:0/3119952797 >> v2:172.21.15.177:3300/0 conn(0x7f3088095710 0x7f3088133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:37:08.435 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f308e053640 1 --2- 172.21.15.177:0/3119952797 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 0x7f308806b5b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3084009e50 tx=0x7f308402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4b57b9a5ab934d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f308d051640 1 -- 172.21.15.177:0/3119952797 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084031600 con 0x7f308806b1d0 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f308d051640 1 -- 172.21.15.177:0/3119952797 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308402d050 con 0x7f308806b1d0 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f308d051640 1 -- 172.21.15.177:0/3119952797 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084031980 con 0x7f308806b1d0 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 -- 172.21.15.177:0/3119952797 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 msgr2=0x7f308806b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 --2- 172.21.15.177:0/3119952797 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 0x7f308806b5b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3084009e50 tx=0x7f308402a5f0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 -- 172.21.15.177:0/3119952797 shutdown_connections 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 --2- 172.21.15.177:0/3119952797 >> v2:172.21.15.177:3300/0 conn(0x7f3088095710 0x7f3088133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 --2- 172.21.15.177:0/3119952797 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806b1d0 0x7f308806b5b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 -- 172.21.15.177:0/3119952797 >> 172.21.15.177:0/3119952797 conn(0x7f3088098ba0 msgr2=0x7f3088096470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 -- 172.21.15.177:0/3119952797 shutdown_connections 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.429+0000 7f30902b7640 1 -- 172.21.15.177:0/3119952797 wait complete. 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.430+0000 7f30902b7640 1 Processor -- start 2024-09-13T18:37:08.436 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.430+0000 7f30902b7640 1 -- start start 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 0x7f30881c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 0x7f30881c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3088095710 0x7f3088123820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881259c0 con 0x7f308806b1d0 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308808bea0 con 0x7f3088095710 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f308808b7d0 con 0x7f308806baf0 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308e053640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 0x7f30881c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 0x7f30881c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308e053640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 0x7f30881c07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:37500/0 (socket says 172.21.15.177:37500) 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308e053640 1 -- 172.21.15.177:0/308516380 learned_addr learned my addr 172.21.15.177:0/308516380 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 0x7f30881c0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:39856/0 (socket says 172.21.15.177:39856) 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308e854640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3088095710 0x7f3088123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 -- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3088095710 msgr2=0x7f3088123820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3088095710 0x7f3088123820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 -- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 msgr2=0x7f30881c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 0x7f30881c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 -- 172.21.15.177:0/308516380 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080009400 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308e053640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 0x7f30881c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f308d852640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 0x7f30881c0d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f307800c3e0 tx=0x7f307800c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mon.2 v2:172.21.15.177:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30780074e0 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mon.2 v2:172.21.15.177:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3078007640 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084009b30 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.431+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mon.2 v2:172.21.15.177:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078004a20 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.432+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3088125b50 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.432+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mon.2 v2:172.21.15.177:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3078004b80 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.433+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308809b670 con 0x7f308806baf0 2024-09-13T18:37:08.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.433+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mon.2 v2:172.21.15.177:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f307805c550 con 0x7f308806baf0 2024-09-13T18:37:08.440 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.436+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mon.2 v2:172.21.15.177:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f307803a1c0 con 0x7f308806baf0 2024-09-13T18:37:08.573 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.570+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 --> v1:172.21.15.177:6816/516212916 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3088095ba0 con 0x7f306c047560 2024-09-13T18:37:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.571+0000 7f307effd640 1 -- 172.21.15.177:0/308516380 <== mgr.14106 v1:172.21.15.177:6816/516212916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34912 (unknown 2965378022 0 533671377) 0x7f30840087b0 con 0x7f306c047560 2024-09-13T18:37:08.575 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-13T18:37:08.575 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.572+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 >> v1:172.21.15.177:6816/516212916 conn(0x7f306c047560 legacy=0x7f306c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.572+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 msgr2=0x7f30881c0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.572+0000 7f30902b7640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 0x7f30881c0d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f307800c3e0 tx=0x7f307800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 shutdown_connections 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3088095710 0x7f3088123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f308806baf0 0x7f30881c0d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 --2- 172.21.15.177:0/308516380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f308806b1d0 0x7f30881c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 >> 172.21.15.177:0/308516380 conn(0x7f3088098ba0 msgr2=0x7f30880948d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 shutdown_connections 2024-09-13T18:37:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.573+0000 7f30902b7640 1 -- 172.21.15.177:0/308516380 wait complete. 2024-09-13T18:37:08.587 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-13T18:37:06.598348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":112300,"kb_used_data":2252,"kb_used_omap":17,"kb_used_meta":109998,"kb_avail":377375060,"statfs":{"total":386547056640,"available":386432061440,"internally_reserved":0,"allocated":2306048,"data_stored":1570439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":112638197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002294"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:37:03.930400+0000","last_change":"2024-09-13T18:37:03.930526+0000","last_active":"2024-09-13T18:37:03.930400+0000","last_peered":"2024-09-13T18:37:03.930400+0000","last_clean":"2024-09-13T18:37:03.930400+0000","last_became_active":"2024-09-13T18:36:36.888940+0000","last_became_peered":"2024-09-13T18:36:36.888940+0000","last_unstale":"2024-09-13T18:37:03.930400+0000","last_undegraded":"2024-09-13T18:37:03.930400+0000","last_fullsized":"2024-09-13T18:37:03.930400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:37:03.930378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:37:03.930378+0000","last_clean_scrub_stamp":"2024-09-13T18:37:03.930378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:51.429759+0000","scrub_duration":0.0014931020000000001,"objects_trimmed":0,"snaptrim_duration":0.000848068,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:54.603186+0000","last_change":"2024-09-13T18:36:54.603300+0000","last_active":"2024-09-13T18:36:54.603186+0000","last_peered":"2024-09-13T18:36:54.603186+0000","last_clean":"2024-09-13T18:36:54.603186+0000","last_became_active":"2024-09-13T18:36:34.636322+0000","last_became_peered":"2024-09-13T18:36:34.636322+0000","last_unstale":"2024-09-13T18:36:54.603186+0000","last_undegraded":"2024-09-13T18:36:54.603186+0000","last_fullsized":"2024-09-13T18:36:54.603186+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:36:54.603164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:36:54.603164+0000","last_clean_scrub_stamp":"2024-09-13T18:36:54.603164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:28.474905+0000","scrub_duration":0.001483182,"objects_trimmed":0,"snaptrim_duration":0.00091051499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:37:02.934899+0000","last_change":"2024-09-13T18:37:02.935009+0000","last_active":"2024-09-13T18:37:02.934899+0000","last_peered":"2024-09-13T18:37:02.934899+0000","last_clean":"2024-09-13T18:37:02.934899+0000","last_became_active":"2024-09-13T18:36:36.888045+0000","last_became_peered":"2024-09-13T18:36:36.888045+0000","last_unstale":"2024-09-13T18:37:02.934899+0000","last_undegraded":"2024-09-13T18:37:02.934899+0000","last_fullsized":"2024-09-13T18:37:02.934899+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:37:02.934873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:37:02.934873+0000","last_clean_scrub_stamp":"2024-09-13T18:37:02.934873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:42.469107+0000","scrub_duration":0.001665873,"objects_trimmed":0,"snaptrim_duration":0.0010903379999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:52.597971+0000","last_change":"2024-09-13T18:36:52.598095+0000","last_active":"2024-09-13T18:36:52.597971+0000","last_peered":"2024-09-13T18:36:52.597971+0000","last_clean":"2024-09-13T18:36:52.597971+0000","last_became_active":"2024-09-13T18:36:34.634989+0000","last_became_peered":"2024-09-13T18:36:34.634989+0000","last_unstale":"2024-09-13T18:36:52.597971+0000","last_undegraded":"2024-09-13T18:36:52.597971+0000","last_fullsized":"2024-09-13T18:36:52.597971+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:36:52.597948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:36:52.597948+0000","last_clean_scrub_stamp":"2024-09-13T18:36:52.597948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:32.498015+0000","scrub_duration":0.001497398,"objects_trimmed":0,"snaptrim_duration":0.00081751800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"33'3","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:59.872063+0000","last_change":"2024-09-13T18:36:59.872190+0000","last_active":"2024-09-13T18:36:59.872063+0000","last_peered":"2024-09-13T18:36:59.872063+0000","last_clean":"2024-09-13T18:36:59.872063+0000","last_became_active":"2024-09-13T18:36:36.656138+0000","last_became_peered":"2024-09-13T18:36:36.656138+0000","last_unstale":"2024-09-13T18:36:59.872063+0000","last_undegraded":"2024-09-13T18:36:59.872063+0000","last_fullsized":"2024-09-13T18:36:59.872063+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-13T18:36:59.872038+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-13T18:36:59.872038+0000","last_clean_scrub_stamp":"2024-09-13T18:36:59.872038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:40:42.873095+0000","scrub_duration":0.003146818,"objects_trimmed":0,"snaptrim_duration":0.00090742100000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:49.880005+0000","last_change":"2024-09-13T18:36:49.880116+0000","last_active":"2024-09-13T18:36:49.880005+0000","last_peered":"2024-09-13T18:36:49.880005+0000","last_clean":"2024-09-13T18:36:49.880005+0000","last_became_active":"2024-09-13T18:36:36.655800+0000","last_became_peered":"2024-09-13T18:36:36.655800+0000","last_unstale":"2024-09-13T18:36:49.880005+0000","last_undegraded":"2024-09-13T18:36:49.880005+0000","last_fullsized":"2024-09-13T18:36:49.880005+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:36:49.879985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:36:49.879985+0000","last_clean_scrub_stamp":"2024-09-13T18:36:49.879985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:33.773278+0000","scrub_duration":0.0013536139999999999,"objects_trimmed":0,"snaptrim_duration":0.00022599299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:49.831039+0000","last_change":"2024-09-13T18:36:49.831174+0000","last_active":"2024-09-13T18:36:49.831039+0000","last_peered":"2024-09-13T18:36:49.831039+0000","last_clean":"2024-09-13T18:36:49.831039+0000","last_became_active":"2024-09-13T18:36:36.656990+0000","last_became_peered":"2024-09-13T18:36:36.656990+0000","last_unstale":"2024-09-13T18:36:49.831039+0000","last_undegraded":"2024-09-13T18:36:49.831039+0000","last_fullsized":"2024-09-13T18:36:49.831039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:36:49.831017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:36:49.831017+0000","last_clean_scrub_stamp":"2024-09-13T18:36:49.831017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:00:53.978980+0000","scrub_duration":0.001741198,"objects_trimmed":0,"snaptrim_duration":0.00058594699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"33'2","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:53.878543+0000","last_change":"2024-09-13T18:36:53.878647+0000","last_active":"2024-09-13T18:36:53.878543+0000","last_peered":"2024-09-13T18:36:53.878543+0000","last_clean":"2024-09-13T18:36:53.878543+0000","last_became_active":"2024-09-13T18:36:36.656089+0000","last_became_peered":"2024-09-13T18:36:36.656089+0000","last_unstale":"2024-09-13T18:36:53.878543+0000","last_undegraded":"2024-09-13T18:36:53.878543+0000","last_fullsized":"2024-09-13T18:36:53.878543+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-13T18:36:53.878516+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-13T18:36:53.878516+0000","last_clean_scrub_stamp":"2024-09-13T18:36:53.878516+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:03.558426+0000","scrub_duration":0.002900428,"objects_trimmed":0,"snaptrim_duration":0.00060857499999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"33'64","reported_seq":149,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T18:36:50.603291+0000","last_change":"2024-09-13T18:36:50.603402+0000","last_active":"2024-09-13T18:36:50.603291+0000","last_peered":"2024-09-13T18:36:50.603291+0000","last_clean":"2024-09-13T18:36:50.603291+0000","last_became_active":"2024-09-13T18:36:34.636339+0000","last_became_peered":"2024-09-13T18:36:34.636339+0000","last_unstale":"2024-09-13T18:36:50.603291+0000","last_undegraded":"2024-09-13T18:36:50.603291+0000","last_fullsized":"2024-09-13T18:36:50.603291+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'64","last_scrub_stamp":"2024-09-13T18:36:50.603267+0000","last_deep_scrub":"33'64","last_deep_scrub_stamp":"2024-09-13T18:36:50.603267+0000","last_clean_scrub_stamp":"2024-09-13T18:36:50.603267+0000","objects_scrubbed":2,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:05.174489+0000","scrub_duration":0.0046099510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":29,"seq":124554051592,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":276,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":282624,"data_stored":97588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]}]},{"osd":2,"up_from":28,"seq":120259084296,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":278528,"data_stored":97569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]}]},{"osd":1,"up_from":27,"seq":115964117000,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":856,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":876544,"data_stored":687956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":28110398},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":0,"up_from":26,"seq":111669149704,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":868352,"data_stored":687326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":28175933},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:37:08.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.682+0000 7f0b171a0640 1 Processor -- start 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.682+0000 7f0b171a0640 1 -- start start 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b171a0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0b10097ce0 0x7f0b1006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b171a0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 0x7f0b10135350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b171a0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b10099ef0 con 0x7f0b1006c860 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b171a0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b10066270 con 0x7f0b10097ce0 2024-09-13T18:37:08.687 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b171a0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1006da70 con 0x7f0b1006cc70 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1699f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 0x7f0b10135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1619e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0b10097ce0 0x7f0b1006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1699f640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 0x7f0b10135350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:39864/0 (socket says 172.21.15.177:39864) 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1699f640 1 -- 172.21.15.177:0/429190591 learned_addr learned my addr 172.21.15.177:0/429190591 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1599d640 1 --1- 172.21.15.177:0/429190591 >> v1:172.21.15.177:6789/0 conn(0x7f0b1006c860 0x7f0b10131c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:47692/0 (socket says 172.21.15.177:47692) 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1619e640 1 -- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 msgr2=0x7f0b10135350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1619e640 1 --2- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 0x7f0b10135350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1619e640 1 -- 172.21.15.177:0/429190591 >> v1:172.21.15.177:6789/0 conn(0x7f0b1006c860 legacy=0x7f0b10131c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1619e640 1 -- 172.21.15.177:0/429190591 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1006d120 con 0x7f0b10097ce0 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1519c640 1 -- 172.21.15.177:0/429190591 <== mon.0 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277910307 0 0) 0x7f0b10099ef0 con 0x7f0b1006c860 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.683+0000 7f0b1699f640 1 --2- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 0x7f0b10135350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:37:08.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.684+0000 7f0b1619e640 1 --2- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b10097ce0 0x7f0b1006c320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0097a0 tx=0x7f0b0c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8959a0b711287254 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:08.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.684+0000 7f0b1519c640 1 -- 172.21.15.177:0/429190591 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c02adb0 con 0x7f0b10097ce0 2024-09-13T18:37:08.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.684+0000 7f0b1519c640 1 -- 172.21.15.177:0/429190591 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c032440 con 0x7f0b10097ce0 2024-09-13T18:37:08.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.684+0000 7f0b1519c640 1 -- 172.21.15.177:0/429190591 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0327d0 con 0x7f0b10097ce0 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 -- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b10097ce0 msgr2=0x7f0b1006c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 --2- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b10097ce0 0x7f0b1006c320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0097a0 tx=0x7f0b0c02a060 comp rx=0 tx=0).stop 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 -- 172.21.15.177:0/429190591 shutdown_connections 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 --2- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006cc70 0x7f0b10135350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 --2- 172.21.15.177:0/429190591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b10097ce0 0x7f0b1006c320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 -- 172.21.15.177:0/429190591 >> 172.21.15.177:0/429190591 conn(0x7f0b1009a9b0 msgr2=0x7f0b1009adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 -- 172.21.15.177:0/429190591 shutdown_connections 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 -- 172.21.15.177:0/429190591 wait complete. 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.685+0000 7f0b171a0640 1 Processor -- start 2024-09-13T18:37:08.692 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 -- start start 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 0x7f0b10122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 0x7f0b10123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b10097ce0 0x7f0b10123850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10125080 con 0x7f0b1006cc70 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10125280 con 0x7f0b10097ce0 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b171a0640 1 -- --> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b100715c0 con 0x7f0b1006c860 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 0x7f0b10123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1619e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 0x7f0b10122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 0x7f0b10123310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:37504/0 (socket says 172.21.15.177:37504) 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1619e640 1 --2- >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 0x7f0b10122dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3302/0 says I am v2:172.21.15.177:39876/0 (socket says 172.21.15.177:39876) 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 -- 172.21.15.177:0/2661974585 learned_addr learned my addr 172.21.15.177:0/2661974585 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-13T18:37:08.693 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1699f640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b10097ce0 0x7f0b10123850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 -- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 msgr2=0x7f0b10122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 0x7f0b10122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 -- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b10097ce0 msgr2=0x7f0b10123850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b10097ce0 0x7f0b10123850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 -- 172.21.15.177:0/2661974585 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08009b30 con 0x7f0b1006cc70 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1699f640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b10097ce0 0x7f0b10123850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1619e640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 0x7f0b10122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.686+0000 7f0b1599d640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 0x7f0b10123310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000c4d0 tx=0x7f0b0000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.687+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b000074e0 con 0x7f0b1006cc70 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.687+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b00007640 con 0x7f0b1006cc70 2024-09-13T18:37:08.694 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.687+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c009400 con 0x7f0b1006cc70 2024-09-13T18:37:08.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.687+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00004a20 con 0x7f0b1006cc70 2024-09-13T18:37:08.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.687+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b10124800 con 0x7f0b1006cc70 2024-09-13T18:37:08.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.688+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0b00004b80 con 0x7f0b1006cc70 2024-09-13T18:37:08.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.688+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b10125a10 con 0x7f0b1006cc70 2024-09-13T18:37:08.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.688+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f0b0005be10 con 0x7f0b1006cc70 2024-09-13T18:37:08.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.691+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0b10125a10 con 0x7f0b1006cc70 2024-09-13T18:37:08.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.827+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0b10125a10 con 0x7f0b1006cc70 2024-09-13T18:37:08.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.833+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0b00039a80 con 0x7f0b1006cc70 2024-09-13T18:37:08.836 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.833+0000 7f0b06ffd640 1 -- 172.21.15.177:0/2661974585 <== mon.0 v2:172.21.15.177:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0b10125a10 con 0x7f0b1006cc70 2024-09-13T18:37:08.838 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 >> v1:172.21.15.177:6816/516212916 conn(0x7f0af4047560 legacy=0x7f0af4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 msgr2=0x7f0b10123310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 0x7f0b10123310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000c4d0 tx=0x7f0b0000c990 comp rx=0 tx=0).stop 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 shutdown_connections 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b10097ce0 0x7f0b10123850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b1006cc70 0x7f0b10123310 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 --2- 172.21.15.177:0/2661974585 >> [v2:172.21.15.177:3302/0,v1:172.21.15.177:6791/0] conn(0x7f0b1006c860 0x7f0b10122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 >> 172.21.15.177:0/2661974585 conn(0x7f0b1009a9b0 msgr2=0x7f0b1006fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 shutdown_connections 2024-09-13T18:37:08.839 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-13T18:37:08.835+0000 7f0b171a0640 1 -- 172.21.15.177:0/2661974585 wait complete. 2024-09-13T18:37:08.850 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T18:37:08.850 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T18:37:08.850 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T18:37:08.850 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:08.885 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T18:37:08.885 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T18:37:08.885 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:08.952 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T18:37:08.952 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T18:37:08.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.022 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T18:37:09.022 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T18:37:09.022 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.091 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T18:37:09.091 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T18:37:09.091 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T18:37:09.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.157 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T18:37:09.157 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T18:37:09.157 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.218 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T18:37:09.219 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T18:37:09.219 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:37:09.219 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.270 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:37:09.270 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:37:09.270 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.321 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:37:09.321 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:37:09.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:09.372 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:37:09.372 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T18:37:09.373 DEBUG:teuthology.orchestra.run.smithi177:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2024-09-13T18:37:09.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:37:09.458 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T18:37:09.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:37:09.599 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T18:37:09.670 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:37:09.671 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T18:37:09.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:37:09.711 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T18:37:09.805 INFO:tasks.ceph:Archiving mon data... 2024-09-13T18:37:09.806 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993/data/mon.a.tgz 2024-09-13T18:37:09.806 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-13T18:37:09.821 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.efXYIRD0q6 2024-09-13T18:37:09.821 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.efXYIRD0q6 2024-09-13T18:37:09.935 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.efXYIRD0q6 2024-09-13T18:37:10.017 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.efXYIRD0q6 is 213KB 2024-09-13T18:37:10.084 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.efXYIRD0q6 2024-09-13T18:37:10.099 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993/data/mon.b.tgz 2024-09-13T18:37:10.099 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-13T18:37:10.153 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.TujPFP1zpI 2024-09-13T18:37:10.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TujPFP1zpI 2024-09-13T18:37:10.265 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.TujPFP1zpI 2024-09-13T18:37:10.348 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.TujPFP1zpI is 250KB 2024-09-13T18:37:10.444 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.TujPFP1zpI 2024-09-13T18:37:10.458 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993/data/mon.c.tgz 2024-09-13T18:37:10.458 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-13T18:37:10.512 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.nw8Xn1TBfS 2024-09-13T18:37:10.513 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.nw8Xn1TBfS 2024-09-13T18:37:10.624 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.nw8Xn1TBfS 2024-09-13T18:37:10.707 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.nw8Xn1TBfS is 250KB 2024-09-13T18:37:10.801 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.nw8Xn1TBfS 2024-09-13T18:37:10.816 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T18:37:10.817 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:37:10.880 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T18:37:10.880 DEBUG:teuthology.orchestra.run.smithi177:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:37:10.946 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:37:10.950 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T18:37:10.952 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993/remote/smithi177/crash 2024-09-13T18:37:10.953 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:37:11.015 INFO:tasks.ceph:Compressing logs... 2024-09-13T18:37:11.016 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:37:11.605 INFO:tasks.ceph:Archiving logs... 2024-09-13T18:37:11.606 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993/remote/smithi177/log 2024-09-13T18:37:11.607 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:37:11.802 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T18:37:11.818 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T18:37:11.819 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:37:11.878 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T18:37:11.895 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T18:37:11.895 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:37:11.911 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:37:11.914 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:37:11.915 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-09-13T18:37:11.915 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv01.front.sepia.ceph.com 3 6 177 50 -1130us[-1138us] +/- 55ms 2024-09-13T18:37:11.915 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv02.front.sepia.ceph.com 3 6 177 50 +2865us[+2858us] +/- 73ms 2024-09-13T18:37:11.915 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 50 -1357us[-1365us] +/- 74ms 2024-09-13T18:37:11.915 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:37:11.916 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T18:37:11.927 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T18:37:11.927 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T18:37:11.938 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T18:37:12.020 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T18:37:12.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:37:12.047 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T18:37:12.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T18:37:12.070 INFO:teuthology.task.internal:Duration was 473.603004 seconds 2024-09-13T18:37:12.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T18:37:12.081 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T18:37:12.082 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:37:12.126 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:37:12.389 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T18:37:12.390 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-09-13T18:37:12.390 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:37:12.449 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T18:37:12.450 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:37:12.516 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T18:37:12.516 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:37:12.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T18:37:12.679 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T18:37:12.679 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:37:12.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T18:37:12.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:37:12.769 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-09-13T18:37:12.798 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:37:12.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:37:12.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T18:37:12.864 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T18:37:12.865 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903993/remote/smithi177 2024-09-13T18:37:12.865 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:37:12.927 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T18:37:12.928 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:37:12.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T18:37:12.991 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T18:37:12.991 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T18:37:13.004 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T18:37:13.005 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:37:13.032 INFO:teuthology.orchestra.run.smithi177.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:37 /home/ubuntu/cephtest 2024-09-13T18:37:13.034 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T18:37:13.045 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T18:37:13.074 INFO:teuthology.run:Summary data: description: fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} duration: 473.6030044555664 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T18:37:13.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:37:13.164 INFO:teuthology.run:pass