2024-04-25T10:48:32.449 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:48:32.449 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-pacific 1-upgrade}} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106 --verbose -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/orig.config.yaml 2024-04-25T10:48:32.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:48:32.587 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-pacific 1-upgrade}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673106' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/pacific latest ' name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 targets: smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNUdPMgK0bmnK9jNsfG0O+98ZmMNmpdioLaWkqOf26OKVZV9I3/MfmU12C9kvu406Jd8OS289V/TYxvRcsELEQ= tasks: - install: branch: pacific exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing pacific' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client pacific - print: '**** done ceph' - print: '*** upgrading, no cephfs present' - exec: mon.a: - ceph fs dump - install.upgrade: mon.a: branch: reef - print: '**** done install.upgrade' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph versions - ceph osd dump -f json-pretty - ceph fs dump - ceph osd require-osd-release quincy - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:48:32.587 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:48:32.588 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:48:32.588 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:48:32.589 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:48:32.614 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:48:32.614 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:48:32.614 INFO:teuthology.packaging:ref: None 2024-04-25T10:48:32.614 INFO:teuthology.packaging:tag: None 2024-04-25T10:48:32.614 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:48:32.614 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:48:32.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:48:32.847 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:48:32.849 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:48:32.857 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:48:32.857 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:48:32.869 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:48:32.886 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:48:32.898 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:48:32.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:43:03.864061', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNUdPMgK0bmnK9jNsfG0O+98ZmMNmpdioLaWkqOf26OKVZV9I3/MfmU12C9kvu406Jd8OS289V/TYxvRcsELEQ='} 2024-04-25T10:48:32.923 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:48:32.934 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-04-25T10:48:32.935 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:48:32.984 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8841991ee0>, signals=[15]) 2024-04-25T10:48:32.984 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:48:32.996 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:48:32.996 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T10:48:32.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:48:33.077 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:48:33.089 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-04-25T10:48:33.110 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-04-25T10:48:33.111 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-04-25T10:48:33.171 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:48:33.172 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-25T10:48:33.173 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:48:33.173 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-04-25T10:48:33.192 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:48:33.206 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:48:33.220 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:48:33.220 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:48:33.240 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:48:33.248 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:48:33.248 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:48:33.305 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:48:33.346 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:48:33.368 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T10:48:33.368 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T10:48:33.369 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:48:33.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:48:33.389 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-04-25T10:48:33.449 INFO:teuthology.orchestra.run.smithi155.stdout:4.18.0-489.el8.x86_64 2024-04-25T10:48:33.450 INFO:teuthology.task.kernel:Running kernel on smithi155: 4.18.0-489.el8.x86_64 2024-04-25T10:48:33.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum install -y kernel 2024-04-25T10:48:36.829 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - AppStream 29 MB/s | 28 MB 00:00 2024-04-25T10:48:42.338 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - BaseOS 21 MB/s | 10 MB 00:00 2024-04-25T10:48:44.346 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - Extras 45 kB/s | 18 kB 00:00 2024-04-25T10:48:44.638 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - Extras common packages 42 kB/s | 7.7 kB 00:00 2024-04-25T10:48:45.007 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2024-04-25T10:48:57.015 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 16 MB 00:11 2024-04-25T10:49:00.807 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 536 kB/s | 24 kB 00:00 2024-04-25T10:49:03.878 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-25T10:49:03.878 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-25T10:49:04.017 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-25T10:49:04.018 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout:Install 3 Packages 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 90 M 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 96 M 2024-04-25T10:49:04.019 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:49:06.249 INFO:teuthology.orchestra.run.smithi155.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 25 MB/s | 10 MB 00:00 2024-04-25T10:49:07.099 INFO:teuthology.orchestra.run.smithi155.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 29 MB/s | 36 MB 00:01 2024-04-25T10:49:07.499 INFO:teuthology.orchestra.run.smithi155.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 26 MB/s | 44 MB 00:01 2024-04-25T10:49:07.499 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:49:07.500 INFO:teuthology.orchestra.run.smithi155.stdout:Total 26 MB/s | 90 MB 00:03 2024-04-25T10:49:08.189 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:49:08.621 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:49:08.621 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:49:09.860 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:49:09.878 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:49:12.673 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:49:13.216 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-25T10:49:16.757 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-25T10:49:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-25T10:49:20.583 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-25T10:49:20.891 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-25T10:49:42.832 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-25T10:49:42.855 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-25T10:49:43.204 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-25T10:49:43.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-25T10:49:43.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-25T10:49:44.057 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-25T10:49:44.057 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:49:44.057 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:49:44.058 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-25T10:49:44.058 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-25T10:49:44.058 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:49:44.058 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:49:44.208 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:49:44.428 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-25T10:49:44.428 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-25T10:49:44.428 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-25T10:49:44.428 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T10:49:44.428 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T10:49:44.428 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T10:49:44.429 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T10:49:44.429 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:49:44.640 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-25T10:49:44.640 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-25T10:49:44.640 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -qi grub2-tools 2024-04-25T10:49:44.674 INFO:teuthology.orchestra.run.smithi155.stdout:Name : grub2-tools 2024-04-25T10:49:44.674 INFO:teuthology.orchestra.run.smithi155.stdout:Epoch : 1 2024-04-25T10:49:44.674 INFO:teuthology.orchestra.run.smithi155.stdout:Version : 2.02 2024-04-25T10:49:44.674 INFO:teuthology.orchestra.run.smithi155.stdout:Release : 129.el8 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: x86_64 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Group : System Environment/Base 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Size : 9265586 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:License : GPLv3+ 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Relocations : (not relocatable) 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Packager : CentOS BuildSystem 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Vendor : CentOS 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Summary : Support tools for GRUB. 2024-04-25T10:49:44.675 INFO:teuthology.orchestra.run.smithi155.stdout:Description : 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout:hardware devices. 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:49:44.676 INFO:teuthology.orchestra.run.smithi155.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T10:49:44.677 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T10:49:44.677 INFO:teuthology.task.kernel:Updating grub on smithi155 to boot 4.18.0-552.el8.x86_64 2024-04-25T10:49:44.677 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T10:49:45.978 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-04-25T10:49:47.759 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-04-25T10:49:47.760 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-25T10:49:47.793 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.x8MWtZxq4D 2024-04-25T10:49:47.794 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.x8MWtZxq4D 2024-04-25T10:49:47.875 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.x8MWtZxq4D 2024-04-25T10:49:48.042 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.x8MWtZxq4D is 6KB 2024-04-25T10:49:48.090 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.x8MWtZxq4D 2024-04-25T10:49:48.108 DEBUG:teuthology.orchestra.run.smithi155:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T10:49:48.178 INFO:teuthology.orchestra.run.smithi155.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-25T10:49:48.178 INFO:teuthology.orchestra.run.smithi155.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-25T10:49:48.179 INFO:teuthology.orchestra.run.smithi155.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-25T10:49:48.179 INFO:teuthology.orchestra.run.smithi155.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-25T10:49:48.180 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-25T10:49:48.329 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-04-25T10:50:18.361 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T10:50:18.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:50:18.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:50:36.703 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-04-25T10:50:45.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:50:45.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:50:48.767 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-04-25T10:51:00.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:51:00.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:52:00.831 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T10:52:15.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:52:15.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:52:16.133 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-25T10:52:16.436 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:52:16.436 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-552.el8.x86_64"... 2024-04-25T10:52:16.436 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-04-25T10:52:16.455 INFO:teuthology.orchestra.run.smithi155.stdout:4.18.0-552.el8.x86_64 2024-04-25T10:52:16.455 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-25T10:52:16.455 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T10:52:16.455 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T10:52:17.456 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T10:52:17.456 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T10:52:17.526 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-04-25T10:52:17.542 DEBUG:teuthology.parallel:result is None 2024-04-25T10:52:17.543 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:52:17.550 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:52:17.551 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:52:17.571 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:52:17.578 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:52:17.585 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:52:17.585 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:52:17.648 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:52:17.655 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:52:17.655 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:52:17.723 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:52:17.733 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:52:17.734 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:52:17.742 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:52:17.742 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:52:17.803 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:52:17.815 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:52:17.815 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:52:17.862 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:52:17.949 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:52:18.002 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:52:18.083 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:52:18.128 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T10:52:18.128 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:52:18.195 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-04-25T10:52:18.290 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:52:18.437 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:52:18.456 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:52:18.457 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:52:18.482 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:52:18.491 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-25T10:52:18.491 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-04-25T10:52:18.562 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-04-25T10:52:18.564 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:52:18.564 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-04-25T10:52:18.600 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-04-25T10:52:18.601 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-04-25T10:52:18.601 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-25T10:52:18.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:52:18.668 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:52:18.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:52:18.737 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:52:18.747 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:52:18.764 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:52:18.765 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:58:07.146 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-04-25T10:58:07.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:58:07.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:58:07.228 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-25T10:58:07.296 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T10:58:07.297 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:58:07.308 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:58:07.309 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:58:07.309 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:58:07.369 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:58:07.383 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:58:07.452 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-04-25T10:58:07.463 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-04-25T10:58:07.477 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:58:07.490 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:58:07.564 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2024-04-25T10:58:07.568 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:58:07.568 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-04-25T10:58:07.568 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:58:07.569 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:58:07.569 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:58:07.569 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:58:07.570 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:58:07.578 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:58:07.578 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T10:58:07.578 DEBUG:teuthology.task.install:config {'branch': 'pacific', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T10:58:07.578 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:58:07.584 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:58:07.585 INFO:teuthology.task.install:extra packages: ['librados2'] 2024-04-25T10:58:07.585 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:58:07.586 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:58:07.586 INFO:teuthology.packaging:ref: None 2024-04-25T10:58:07.586 INFO:teuthology.packaging:tag: None 2024-04-25T10:58:07.586 INFO:teuthology.packaging:branch: pacific 2024-04-25T10:58:07.586 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:58:07.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-04-25T10:58:07.900 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/ 2024-04-25T10:58:07.901 INFO:teuthology.task.install.rpm:Package version is 16.2.15-0 2024-04-25T10:58:07.935 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:58:07.936 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T10:58:07.936 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:58:07.966 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-04-25T10:58:07.966 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:58:07.966 INFO:teuthology.packaging:ref: None 2024-04-25T10:58:07.966 INFO:teuthology.packaging:tag: None 2024-04-25T10:58:07.966 INFO:teuthology.packaging:branch: pacific 2024-04-25T10:58:07.967 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:58:07.967 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:58:08.034 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:58:08.112 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:58:08.179 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-04-25T10:58:08.180 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-04-25T10:58:08.447 INFO:teuthology.orchestra.run.smithi155.stdout:78 files removed 2024-04-25T10:58:08.475 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw 2024-04-25T10:58:08.994 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 669 kB/s | 76 kB 00:00 2024-04-25T10:58:09.145 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 135 kB/s | 15 kB 00:00 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 13 kB/s | 1.7 kB 00:00 2024-04-25T10:58:10.462 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - AppStream 25 MB/s | 28 MB 00:01 2024-04-25T10:58:16.193 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - BaseOS 15 MB/s | 10 MB 00:00 2024-04-25T10:58:18.155 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - Extras 68 kB/s | 18 kB 00:00 2024-04-25T10:58:18.626 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 7.7 kB 00:00 2024-04-25T10:58:20.224 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - PowerTools 3.3 MB/s | 5.0 MB 00:01 2024-04-25T10:58:21.300 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-04-25T10:58:22.431 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2024-04-25T10:58:26.258 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 296 kB/s | 24 kB 00:00 2024-04-25T10:58:29.876 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:58:29.878 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:58:29.878 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:16.2.15-0.el8 ceph 12 M 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:16.2.15-0.el8 ceph 3.7 M 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:16.2.15-0.el8 ceph 4.2 M 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:16.2.15-0.el8 ceph 6.1 M 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:16.2.15-0.el8 ceph 24 M 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:16.2.15-0.el8 ceph 26 k 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:16.2.15-0.el8 ceph 825 k 2024-04-25T10:58:29.879 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:16.2.15-0.el8 ceph 494 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:16.2.15-0.el8 ceph 3.8 M 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:16.2.15-0.el8 ceph 45 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:16.2.15-0.el8 ceph 99 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:16.2.15-0.el8 ceph 215 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:16.2.15-0.el8 ceph 389 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:16.2.15-0.el8 ceph 369 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:16.2.15-0.el8 ceph 114 k 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:58:29.880 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:58:29.881 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:58:29.881 INFO:teuthology.orchestra.run.smithi155.stdout:Install 21 Packages 2024-04-25T10:58:29.881 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 2 Packages 2024-04-25T10:58:29.881 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:58:29.881 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 58 M 2024-04-25T10:58:29.881 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:58:30.420 INFO:teuthology.orchestra.run.smithi155.stdout:(1/23): ceph-base-16.2.15-0.el8.x86_64.rpm 21 MB/s | 6.1 MB 00:00 2024-04-25T10:58:30.811 INFO:teuthology.orchestra.run.smithi155.stdout:(2/23): ceph-radosgw-16.2.15-0.el8.x86_64.rpm 17 MB/s | 12 MB 00:00 2024-04-25T10:58:30.836 INFO:teuthology.orchestra.run.smithi155.stdout:(3/23): ceph-selinux-16.2.15-0.el8.x86_64.rpm 61 kB/s | 26 kB 00:00 2024-04-25T10:58:30.887 INFO:teuthology.orchestra.run.smithi155.stdout:(4/23): libcephfs2-16.2.15-0.el8.x86_64.rpm 11 MB/s | 825 kB 00:00 2024-04-25T10:58:30.912 INFO:teuthology.orchestra.run.smithi155.stdout:(5/23): libradosstriper1-16.2.15-0.el8.x86_64.r 6.4 MB/s | 494 kB 00:00 2024-04-25T10:58:31.054 INFO:teuthology.orchestra.run.smithi155.stdout:(6/23): librgw2-16.2.15-0.el8.x86_64.rpm 23 MB/s | 3.8 MB 00:00 2024-04-25T10:58:31.080 INFO:teuthology.orchestra.run.smithi155.stdout:(7/23): python3-ceph-argparse-16.2.15-0.el8.x86 269 kB/s | 45 kB 00:00 2024-04-25T10:58:31.105 INFO:teuthology.orchestra.run.smithi155.stdout:(8/23): python3-ceph-common-16.2.15-0.el8.x86_6 1.9 MB/s | 99 kB 00:00 2024-04-25T10:58:31.164 INFO:teuthology.orchestra.run.smithi155.stdout:(9/23): python3-cephfs-16.2.15-0.el8.x86_64.rpm 2.5 MB/s | 215 kB 00:00 2024-04-25T10:58:31.189 INFO:teuthology.orchestra.run.smithi155.stdout:(10/23): python3-rados-16.2.15-0.el8.x86_64.rpm 4.5 MB/s | 389 kB 00:00 2024-04-25T10:58:31.231 INFO:teuthology.orchestra.run.smithi155.stdout:(11/23): python3-rbd-16.2.15-0.el8.x86_64.rpm 5.4 MB/s | 369 kB 00:00 2024-04-25T10:58:31.264 INFO:teuthology.orchestra.run.smithi155.stdout:(12/23): python3-rgw-16.2.15-0.el8.x86_64.rpm 1.5 MB/s | 114 kB 00:00 2024-04-25T10:58:31.557 INFO:teuthology.orchestra.run.smithi155.stdout:(13/23): ceph-common-16.2.15-0.el8.x86_64.rpm 17 MB/s | 24 MB 00:01 2024-04-25T10:58:31.623 INFO:teuthology.orchestra.run.smithi155.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 724 kB/s | 259 kB 00:00 2024-04-25T10:58:31.698 INFO:teuthology.orchestra.run.smithi155.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 310 kB/s | 44 kB 00:00 2024-04-25T10:58:31.732 INFO:teuthology.orchestra.run.smithi155.stdout:(16/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.3 MB/s | 677 kB 00:00 2024-04-25T10:58:31.758 INFO:teuthology.orchestra.run.smithi155.stdout:(17/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 351 kB/s | 47 kB 00:00 2024-04-25T10:58:32.191 INFO:teuthology.orchestra.run.smithi155.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 136 kB/s | 59 kB 00:00 2024-04-25T10:58:32.300 INFO:teuthology.orchestra.run.smithi155.stdout:(19/23): leveldb-1.22-1.el8.x86_64.rpm 319 kB/s | 181 kB 00:00 2024-04-25T10:58:32.375 INFO:teuthology.orchestra.run.smithi155.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 453 kB/s | 306 kB 00:00 2024-04-25T10:58:32.400 INFO:teuthology.orchestra.run.smithi155.stdout:(21/23): libunwind-1.3.1-3.el8.x86_64.rpm 360 kB/s | 75 kB 00:00 2024-04-25T10:58:32.586 INFO:teuthology.orchestra.run.smithi155.stdout:(22/23): librbd1-16.2.15-0.el8.x86_64.rpm 20 MB/s | 4.2 MB 00:00 2024-04-25T10:58:32.651 INFO:teuthology.orchestra.run.smithi155.stdout:(23/23): librados2-16.2.15-0.el8.x86_64.rpm 11 MB/s | 3.7 MB 00:00 2024-04-25T10:58:32.653 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:58:32.653 INFO:teuthology.orchestra.run.smithi155.stdout:Total 21 MB/s | 58 MB 00:02 2024-04-25T10:58:32.696 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:58:32.744 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:58:32.744 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:58:33.094 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:58:33.097 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:58:34.506 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:58:34.722 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-04-25T10:58:34.939 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-25T10:58:35.326 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-25T10:58:35.555 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:16.2.15-0.el8.x86_64 3/25 2024-04-25T10:58:35.650 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.15-0.el8.x86_64 3/25 2024-04-25T10:58:35.957 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:16.2.15-0.el8.x86_64 4/25 2024-04-25T10:58:36.180 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:16.2.15-0.el8.x86_64 5/25 2024-04-25T10:58:36.593 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:16.2.15-0.el8.x86_64 5/25 2024-04-25T10:58:36.789 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:16.2.15-0.el8.x86_64 6/25 2024-04-25T10:58:36.881 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.15-0.el8.x86_64 6/25 2024-04-25T10:58:37.114 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-04-25T10:58:37.624 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-04-25T10:58:37.788 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:16.2.15-0.el8.x86_64 9/25 2024-04-25T10:58:37.862 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:16.2.15-0.el8.x86_64 9/25 2024-04-25T10:58:38.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-04-25T10:58:38.277 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:16.2.15-0.el8.x86_64 11/25 2024-04-25T10:58:38.467 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:16.2.15-0.el8.x86_64 12/25 2024-04-25T10:58:38.714 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:16.2.15-0.el8.x86_64 13/25 2024-04-25T10:58:38.971 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:16.2.15-0.el8.x86_64 14/25 2024-04-25T10:58:39.113 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:16.2.15-0.el8.x86_64 15/25 2024-04-25T10:58:39.183 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:16.2.15-0.el8.x86_64 15/25 2024-04-25T10:58:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-25T10:58:39.638 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-04-25T10:58:39.832 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-25T10:58:40.063 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:16.2.15-0.el8.x86_64 19/25 2024-04-25T10:58:42.535 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:16.2.15-0.el8.x86_64 20/25 2024-04-25T10:58:42.746 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:16.2.15-0.el8.x86_64 20/25 2024-04-25T10:58:43.467 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:16.2.15-0.el8.x86_64 20/25 2024-04-25T10:58:43.745 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:16.2.15-0.el8.x86_64 21/25 2024-04-25T10:58:43.901 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:16.2.15-0.el8.x86_64 21/25 2024-04-25T10:58:44.138 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:16.2.15-0.el8.x86_64 22/25 2024-04-25T10:59:03.347 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:16.2.15-0.el8.x86_64 22/25 2024-04-25T10:59:03.348 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-04-25T10:59:03.348 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-04-25T10:59:03.348 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2024-04-25T10:59:03.348 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /run 2024-04-25T10:59:03.348 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-04-25T10:59:03.349 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-04-25T10:59:03.349 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-04-25T10:59:03.349 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-04-25T10:59:03.349 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2024-04-25T10:59:03.349 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:04.517 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:16.2.15-0.el8.x86_64 23/25 2024-04-25T10:59:04.594 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:16.2.15-0.el8.x86_64 23/25 2024-04-25T10:59:04.594 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-25T10:59:04.836 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-25T10:59:04.836 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:16.2.15-0.el8.x86_64 1/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:16.2.15-0.el8.x86_64 2/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:16.2.15-0.el8.x86_64 3/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:16.2.15-0.el8.x86_64 4/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:16.2.15-0.el8.x86_64 5/25 2024-04-25T10:59:05.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:16.2.15-0.el8.x86_64 6/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:16.2.15-0.el8.x86_64 7/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:16.2.15-0.el8.x86_64 8/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:16.2.15-0.el8.x86_64 9/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:16.2.15-0.el8.x86_64 10/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:16.2.15-0.el8.x86_64 11/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:16.2.15-0.el8.x86_64 12/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:16.2.15-0.el8.x86_64 13/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-25T10:59:05.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.15-0.el8.x86_64 22/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-25T10:59:05.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.15-0.el8.x86_64 24/25 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:16.2.15-0.el8.x86_64 librbd1-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: leveldb-1.22-1.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.553 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.554 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-25T10:59:06.554 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.554 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.554 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:06.554 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:06.554 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:06.881 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-test 2024-04-25T10:59:07.613 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:41 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:09.261 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:09.262 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:09.262 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:16.2.15-0.el8 ceph 52 M 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:16.2.15-0.el8 ceph 174 k 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-25T10:59:09.263 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 53 M 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 197 M 2024-04-25T10:59:09.264 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:59:09.587 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 1.4 MB/s | 203 kB 00:00 2024-04-25T10:59:09.638 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): oniguruma-6.8.2-3.el8.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T10:59:09.687 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-04-25T10:59:09.729 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 71 kB 00:00 2024-04-25T10:59:09.780 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): libcephsqlite-16.2.15-0.el8.x86_64.rpm 524 kB/s | 174 kB 00:00 2024-04-25T10:59:11.072 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): ceph-test-16.2.15-0.el8.x86_64.rpm 32 MB/s | 52 MB 00:01 2024-04-25T10:59:11.073 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:59:11.074 INFO:teuthology.orchestra.run.smithi155.stdout:Total 29 MB/s | 53 MB 00:01 2024-04-25T10:59:11.116 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:59:11.137 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:59:11.137 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:59:11.474 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:59:11.477 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:59:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:59:12.188 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-25T10:59:12.414 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-25T10:59:12.525 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-25T10:59:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-25T10:59:12.797 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-25T10:59:12.933 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:16.2.15-0.el8.x86_64 5/6 2024-04-25T10:59:17.550 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:16.2.15-0.el8.x86_64 5/6 2024-04-25T10:59:17.806 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:16.2.15-0.el8.x86_64 6/6 2024-04-25T10:59:18.090 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-test-2:16.2.15-0.el8.x86_64 6/6 2024-04-25T10:59:18.091 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:16.2.15-0.el8.x86_64 1/6 2024-04-25T10:59:18.091 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:16.2.15-0.el8.x86_64 2/6 2024-04-25T10:59:18.091 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-25T10:59:18.091 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-25T10:59:18.091 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-25T10:59:18.531 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-25T10:59:18.531 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:18.531 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:59:18.532 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:16.2.15-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-04-25T10:59:18.532 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:16.2.15-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-04-25T10:59:18.532 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-04-25T10:59:18.532 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:18.532 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:18.762 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph 2024-04-25T10:59:19.495 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:53 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:21.162 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:16.2.15-0.el8 ceph 6.3 k 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:16.2.15-0.el8 ceph 2.4 M 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:16.2.15-0.el8 ceph 1.6 M 2024-04-25T10:59:21.165 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:16.2.15-0.el8 ceph-noarch 245 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:16.2.15-0.el8 ceph 5.2 M 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:16.2.15-0.el8 ceph 19 M 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-25T10:59:21.166 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-25T10:59:21.167 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout:Install 32 Packages 2024-04-25T10:59:21.168 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:21.169 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 32 M 2024-04-25T10:59:21.169 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 104 M 2024-04-25T10:59:21.169 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:59:21.625 INFO:teuthology.orchestra.run.smithi155.stdout:(1/32): ceph-16.2.15-0.el8.x86_64.rpm 86 kB/s | 6.3 kB 00:00 2024-04-25T10:59:21.768 INFO:teuthology.orchestra.run.smithi155.stdout:(2/32): ceph-mds-16.2.15-0.el8.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T10:59:21.817 INFO:teuthology.orchestra.run.smithi155.stdout:(3/32): ceph-mgr-16.2.15-0.el8.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-25T10:59:22.009 INFO:teuthology.orchestra.run.smithi155.stdout:(4/32): ceph-mon-16.2.15-0.el8.x86_64.rpm 14 MB/s | 5.2 MB 00:00 2024-04-25T10:59:22.126 INFO:teuthology.orchestra.run.smithi155.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-04-25T10:59:22.168 INFO:teuthology.orchestra.run.smithi155.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 925 kB/s | 39 kB 00:00 2024-04-25T10:59:22.218 INFO:teuthology.orchestra.run.smithi155.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 2.0 MB/s | 103 kB 00:00 2024-04-25T10:59:22.261 INFO:teuthology.orchestra.run.smithi155.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 1.3 MB/s | 57 kB 00:00 2024-04-25T10:59:22.327 INFO:teuthology.orchestra.run.smithi155.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 6.7 MB/s | 457 kB 00:00 2024-04-25T10:59:22.361 INFO:teuthology.orchestra.run.smithi155.stdout:(10/32): ceph-mgr-modules-core-16.2.15-0.el8.no 450 kB/s | 245 kB 00:00 2024-04-25T10:59:22.620 INFO:teuthology.orchestra.run.smithi155.stdout:(11/32): ceph-osd-16.2.15-0.el8.x86_64.rpm 22 MB/s | 19 MB 00:00 2024-04-25T10:59:22.645 INFO:teuthology.orchestra.run.smithi155.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 783 kB/s | 249 kB 00:00 2024-04-25T10:59:22.762 INFO:teuthology.orchestra.run.smithi155.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 769 kB/s | 109 kB 00:00 2024-04-25T10:59:22.821 INFO:teuthology.orchestra.run.smithi155.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 1.4 MB/s | 641 kB 00:00 2024-04-25T10:59:22.913 INFO:teuthology.orchestra.run.smithi155.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 163 kB/s | 44 kB 00:00 2024-04-25T10:59:23.105 INFO:teuthology.orchestra.run.smithi155.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 542 kB/s | 185 kB 00:00 2024-04-25T10:59:23.164 INFO:teuthology.orchestra.run.smithi155.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.5 MB/s | 384 kB 00:00 2024-04-25T10:59:23.189 INFO:teuthology.orchestra.run.smithi155.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 476 kB/s | 173 kB 00:00 2024-04-25T10:59:23.239 INFO:teuthology.orchestra.run.smithi155.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 233 kB/s | 18 kB 00:00 2024-04-25T10:59:23.273 INFO:teuthology.orchestra.run.smithi155.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 582 kB/s | 49 kB 00:00 2024-04-25T10:59:23.298 INFO:teuthology.orchestra.run.smithi155.stdout:(21/32): python3-jaraco-6.2-6.el8.noarch.rpm 57 kB/s | 11 kB 00:00 2024-04-25T10:59:23.323 INFO:teuthology.orchestra.run.smithi155.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 702 kB/s | 59 kB 00:00 2024-04-25T10:59:23.374 INFO:teuthology.orchestra.run.smithi155.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2024-04-25T10:59:23.399 INFO:teuthology.orchestra.run.smithi155.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 248 kB/s | 19 kB 00:00 2024-04-25T10:59:23.424 INFO:teuthology.orchestra.run.smithi155.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2024-04-25T10:59:23.449 INFO:teuthology.orchestra.run.smithi155.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 317 kB/s | 24 kB 00:00 2024-04-25T10:59:23.504 INFO:teuthology.orchestra.run.smithi155.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 279 kB/s | 29 kB 00:00 2024-04-25T10:59:23.533 INFO:teuthology.orchestra.run.smithi155.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 252 kB/s | 27 kB 00:00 2024-04-25T10:59:23.600 INFO:teuthology.orchestra.run.smithi155.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.6 MB/s | 251 kB 00:00 2024-04-25T10:59:23.626 INFO:teuthology.orchestra.run.smithi155.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.4 MB/s | 255 kB 00:00 2024-04-25T10:59:23.651 INFO:teuthology.orchestra.run.smithi155.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 730 kB/s | 85 kB 00:00 2024-04-25T10:59:23.676 INFO:teuthology.orchestra.run.smithi155.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 305 kB/s | 23 kB 00:00 2024-04-25T10:59:23.680 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:59:23.681 INFO:teuthology.orchestra.run.smithi155.stdout:Total 13 MB/s | 32 MB 00:02 2024-04-25T10:59:23.761 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:59:23.801 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:59:24.120 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:59:24.123 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:59:25.264 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:59:25.502 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-25T10:59:25.747 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-25T10:59:25.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-25T10:59:26.265 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-25T10:59:26.460 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-25T10:59:26.656 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-25T10:59:26.850 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-25T10:59:27.044 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-25T10:59:27.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-25T10:59:27.548 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-25T10:59:27.804 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-25T10:59:28.045 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-25T10:59:28.418 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-25T10:59:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-25T10:59:28.888 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-25T10:59:29.137 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-25T10:59:29.348 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-25T10:59:29.674 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-25T10:59:29.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-25T10:59:30.074 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-25T10:59:30.277 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-25T10:59:30.520 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-25T10:59:30.767 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-25T10:59:31.015 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-25T10:59:31.345 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-25T10:59:31.691 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-25T10:59:32.075 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 27/32 2024-04-25T10:59:32.312 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:16.2.15-0.el8.x86_64 28/32 2024-04-25T10:59:33.851 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:16.2.15-0.el8.x86_64 28/32 2024-04-25T10:59:34.063 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:16.2.15-0.el8.x86_64 29/32 2024-04-25T10:59:34.511 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:16.2.15-0.el8.x86_64 29/32 2024-04-25T10:59:34.713 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:16.2.15-0.el8.x86_64 30/32 2024-04-25T10:59:34.931 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:16.2.15-0.el8.x86_64 30/32 2024-04-25T10:59:35.146 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:16.2.15-0.el8.x86_64 31/32 2024-04-25T10:59:35.165 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:16.2.15-0.el8.x86_64 31/32 2024-04-25T10:59:35.263 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:16.2.15-0.el8.x86_64 32/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-2:16.2.15-0.el8.x86_64 32/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:16.2.15-0.el8.x86_64 1/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:16.2.15-0.el8.x86_64 2/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:16.2.15-0.el8.x86_64 3/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:16.2.15-0.el8.x86_64 4/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:16.2.15-0.el8.x86_64 5/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 6/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-25T10:59:35.682 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-25T10:59:35.683 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-25T10:59:35.684 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-25T10:59:35.684 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-25T10:59:35.684 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-25T10:59:35.685 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-25T10:59:35.685 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-25T10:59:35.685 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-25T10:59:35.685 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-25T10:59:36.542 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-25T10:59:36.542 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:36.542 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-25T10:59:36.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-2.6-1.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-25T10:59:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-25T10:59:36.545 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-25T10:59:36.545 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-25T10:59:36.545 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-25T10:59:36.545 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-25T10:59:36.545 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:36.545 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:36.815 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-base 2024-04-25T10:59:37.579 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:11 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:39.169 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-base-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T10:59:39.220 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:39.221 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T10:59:39.221 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:39.331 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T10:59:40.087 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:14 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:41.749 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:16.2.15-0.el8 ceph 161 k 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:41.750 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T10:59:41.751 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:41.751 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 161 k 2024-04-25T10:59:41.751 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 470 k 2024-04-25T10:59:41.751 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:59:41.819 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-immutable-object-cache-16.2.15-0.el8.x86_6 2.3 MB/s | 161 kB 00:00 2024-04-25T10:59:41.820 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:59:41.820 INFO:teuthology.orchestra.run.smithi155.stdout:Total 2.3 MB/s | 161 kB 00:00 2024-04-25T10:59:41.820 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:59:41.830 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:59:41.831 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:59:41.877 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:59:41.880 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:59:41.995 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:59:42.187 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T10:59:42.587 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T10:59:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T10:59:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:43.000 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:59:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:43.000 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:43.218 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr 2024-04-25T10:59:43.981 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:17 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:45.603 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-mgr-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T10:59:45.647 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:45.648 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T10:59:45.648 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:45.753 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-fuse 2024-04-25T10:59:46.503 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:20 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:48.146 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:48.147 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:48.147 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T10:59:48.147 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:48.147 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:59:48.147 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:16.2.15-0.el8 ceph 911 k 2024-04-25T10:59:48.147 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout:Install 2 Packages 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 994 k 2024-04-25T10:59:48.148 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 2.8 M 2024-04-25T10:59:48.149 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:59:48.332 INFO:teuthology.orchestra.run.smithi155.stdout:(1/2): ceph-fuse-16.2.15-0.el8.x86_64.rpm 6.8 MB/s | 911 kB 00:00 2024-04-25T10:59:48.356 INFO:teuthology.orchestra.run.smithi155.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 535 kB/s | 83 kB 00:00 2024-04-25T10:59:48.357 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:59:48.357 INFO:teuthology.orchestra.run.smithi155.stdout:Total 4.6 MB/s | 994 kB 00:00 2024-04-25T10:59:48.367 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:59:48.387 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:59:48.388 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:59:48.470 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:59:48.473 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:59:48.639 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:59:48.901 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-25T10:59:49.085 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:16.2.15-0.el8.x86_64 2/2 2024-04-25T10:59:49.487 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-fuse-2:16.2.15-0.el8.x86_64 2/2 2024-04-25T10:59:49.487 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:16.2.15-0.el8.x86_64 1/2 2024-04-25T10:59:50.334 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-25T10:59:50.334 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:50.334 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:59:50.334 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:16.2.15-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-25T10:59:50.334 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:50.334 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:51.297 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados-devel 2024-04-25T10:59:52.068 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:26 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:53.718 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:53.719 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:53.719 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:16.2.15-0.el8 ceph 131 k 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 131 k 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 432 k 2024-04-25T10:59:53.720 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:59:53.787 INFO:teuthology.orchestra.run.smithi155.stdout:librados-devel-16.2.15-0.el8.x86_64.rpm 1.9 MB/s | 131 kB 00:00 2024-04-25T10:59:53.788 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:59:53.788 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.9 MB/s | 131 kB 00:00 2024-04-25T10:59:53.788 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:59:53.793 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:59:53.793 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:59:53.840 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:59:53.843 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T10:59:53.947 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T10:59:54.150 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T10:59:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados-devel-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T10:59:54.811 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T10:59:54.811 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:54.812 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T10:59:54.812 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:16.2.15-0.el8.x86_64 2024-04-25T10:59:54.812 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:54.812 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:55.008 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs2 2024-04-25T10:59:55.767 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:29 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:57.364 INFO:teuthology.orchestra.run.smithi155.stdout:Package libcephfs2-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T10:59:57.412 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:57.413 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T10:59:57.413 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T10:59:57.513 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs-devel 2024-04-25T10:59:58.270 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T10:59:59.894 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T10:59:59.894 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:16.2.15-0.el8 ceph 25 k 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T10:59:59.895 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T10:59:59.896 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 25 k 2024-04-25T10:59:59.896 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 107 k 2024-04-25T10:59:59.896 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T10:59:59.982 INFO:teuthology.orchestra.run.smithi155.stdout:libcephfs-devel-16.2.15-0.el8.x86_64.rpm 294 kB/s | 25 kB 00:00 2024-04-25T10:59:59.983 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:59:59.983 INFO:teuthology.orchestra.run.smithi155.stdout:Total 290 kB/s | 25 kB 00:00 2024-04-25T10:59:59.983 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T10:59:59.987 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T10:59:59.987 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T10:59:59.989 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T10:59:59.992 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T11:00:00.049 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T11:00:00.207 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:00.469 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs-devel-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:00.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:00.810 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:00.811 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T11:00:00.811 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:16.2.15-0.el8.x86_64 2024-04-25T11:00:00.811 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:00.811 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:00.925 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados2 2024-04-25T11:00:01.684 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:35 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:03.285 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:03.333 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:03.335 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:03.335 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:03.433 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librbd1 2024-04-25T11:00:04.188 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:05.790 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:05.838 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:05.839 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:05.839 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:05.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rados 2024-04-25T11:00:06.695 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:40 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:08.264 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rados-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:08.311 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:08.312 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:08.313 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:08.437 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rgw 2024-04-25T11:00:09.197 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:43 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:10.812 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rgw-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:10.861 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:10.862 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:10.862 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:10.973 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-cephfs 2024-04-25T11:00:11.719 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:45 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:13.326 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-cephfs-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:13.373 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:13.374 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:13.374 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:13.478 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rbd 2024-04-25T11:00:14.238 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:48 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:15.838 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rbd-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:15.885 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:15.886 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:15.886 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:15.987 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-fuse 2024-04-25T11:00:16.742 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:50 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:18.383 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:16.2.15-0.el8 ceph 88 k 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T11:00:18.384 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:18.385 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 88 k 2024-04-25T11:00:18.385 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 240 k 2024-04-25T11:00:18.385 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T11:00:18.471 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-fuse-16.2.15-0.el8.x86_64.rpm 1.0 MB/s | 88 kB 00:00 2024-04-25T11:00:18.471 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:00:18.472 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.0 MB/s | 88 kB 00:00 2024-04-25T11:00:18.472 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T11:00:18.481 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T11:00:18.481 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T11:00:18.528 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T11:00:18.531 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T11:00:18.630 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T11:00:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:19.094 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-fuse-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:19.527 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:19.527 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:19.527 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T11:00:19.527 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:16.2.15-0.el8.x86_64 2024-04-25T11:00:19.528 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:19.528 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:19.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-mirror 2024-04-25T11:00:20.511 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:54 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:22.156 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:16.2.15-0.el8 ceph 3.7 M 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:22.157 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T11:00:22.158 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:22.158 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 3.7 M 2024-04-25T11:00:22.158 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 15 M 2024-04-25T11:00:22.158 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T11:00:22.369 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-mirror-16.2.15-0.el8.x86_64.rpm 17 MB/s | 3.7 MB 00:00 2024-04-25T11:00:22.369 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:00:22.370 INFO:teuthology.orchestra.run.smithi155.stdout:Total 17 MB/s | 3.7 MB 00:00 2024-04-25T11:00:22.370 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T11:00:22.381 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T11:00:22.382 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T11:00:22.441 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T11:00:22.444 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T11:00:22.878 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T11:00:23.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:23.444 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:23.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:23.817 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:23.817 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T11:00:23.818 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:16.2.15-0.el8.x86_64 2024-04-25T11:00:23.818 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:23.818 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:24.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-nbd 2024-04-25T11:00:24.853 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:58 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:26.504 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:26.504 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:26.504 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-04-25T11:00:26.504 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:26.504 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:16.2.15-0.el8 ceph 175 k 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 175 k 2024-04-25T11:00:26.505 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 523 k 2024-04-25T11:00:26.506 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T11:00:26.570 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-nbd-16.2.15-0.el8.x86_64.rpm 2.7 MB/s | 175 kB 00:00 2024-04-25T11:00:26.571 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:00:26.571 INFO:teuthology.orchestra.run.smithi155.stdout:Total 2.6 MB/s | 175 kB 00:00 2024-04-25T11:00:26.571 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T11:00:26.581 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T11:00:26.581 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T11:00:26.627 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T11:00:26.630 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T11:00:26.763 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T11:00:26.978 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:27.299 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-nbd-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:27.667 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:16.2.15-0.el8.x86_64 1/1 2024-04-25T11:00:27.667 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:27.667 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T11:00:27.667 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:16.2.15-0.el8.x86_64 2024-04-25T11:00:27.667 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:27.668 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:27.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados2 2024-04-25T11:00:28.714 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:02:02 ago on Thu 25 Apr 2024 10:58:26 AM UTC. 2024-04-25T11:00:30.315 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.15-0.el8.x86_64 is already installed. 2024-04-25T11:00:30.364 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:00:30.365 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-25T11:00:30.365 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:00:30.471 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:30.472 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:00:30.472 INFO:teuthology.packaging:ref: None 2024-04-25T11:00:30.472 INFO:teuthology.packaging:tag: None 2024-04-25T11:00:30.472 INFO:teuthology.packaging:branch: pacific 2024-04-25T11:00:30.472 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:00:30.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-04-25T11:00:30.791 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:00:30.815 INFO:teuthology.orchestra.run.smithi155.stdout:16.2.15-0.el8 2024-04-25T11:00:30.815 INFO:teuthology.packaging:The installed version of ceph is 16.2.15-0.el8 2024-04-25T11:00:30.815 INFO:teuthology.task.install:The correct ceph version 16.2.15-0 is installed. 2024-04-25T11:00:30.817 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:00:30.817 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:30.817 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:00:30.883 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:00:30.883 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:30.883 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:00:30.950 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:00:31.017 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:00:31.017 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:31.017 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:00:31.084 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:00:31.151 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:00:31.151 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:31.151 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:00:31.218 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:00:31.285 INFO:teuthology.run_tasks:Running task print... 2024-04-25T11:00:31.295 INFO:teuthology.task.print:**** done installing pacific 2024-04-25T11:00:31.295 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T11:00:31.378 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T11:00:31.378 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2024-04-25T11:00:31.405 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T11:00:31.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:00:31.472 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T11:00:31.472 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:00:31.547 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T11:00:31.548 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T11:00:31.548 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-pacific 1-upgrade}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673106', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/pacific latest\n'}], 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'targets': {'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNUdPMgK0bmnK9jNsfG0O+98ZmMNmpdioLaWkqOf26OKVZV9I3/MfmU12C9kvu406Jd8OS289V/TYxvRcsELEQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'pacific', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'print': '**** done installing pacific'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client pacific']}}, {'print': '**** done ceph'}, {'print': '*** upgrading, no cephfs present'}, {'exec': {'mon.a': ['ceph fs dump']}}, {'install.upgrade': {'mon.a': {'branch': 'reef'}}}, {'print': '**** done install.upgrade'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph versions', 'ceph osd dump -f json-pretty', 'ceph fs dump', 'ceph osd require-osd-release quincy', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T11:00:31.548 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:00:31.597 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:00:31.665 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:31.666 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:00:31.723 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:00:31.724 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-04-25T11:00:31.782 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:00:31.782 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:00:31.782 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 166819 Links: 1 2024-04-25T11:00:31.782 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:00:31.782 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:00:31.782 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 11:00:23.200382706 +0000 2024-04-25T11:00:31.783 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 10:57:37.474893383 +0000 2024-04-25T11:00:31.783 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 10:57:37.474893383 +0000 2024-04-25T11:00:31.783 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-04-25T11:00:31.783 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:00:31.847 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T11:00:31.847 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T11:00:31.848 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000256434 s, 2.0 MB/s 2024-04-25T11:00:31.849 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:00:31.908 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-04-25T11:00:31.966 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:00:31.966 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 169090 Links: 1 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 11:00:23.200382706 +0000 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 10:57:37.842886014 +0000 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 10:57:37.842886014 +0000 2024-04-25T11:00:31.967 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-04-25T11:00:31.967 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:00:32.031 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T11:00:32.031 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T11:00:32.032 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.00023987 s, 2.1 MB/s 2024-04-25T11:00:32.032 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:00:32.091 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-04-25T11:00:32.148 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:00:32.148 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 167429 Links: 1 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 11:00:23.200382706 +0000 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 10:57:38.174879366 +0000 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 10:57:38.174879366 +0000 2024-04-25T11:00:32.149 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-04-25T11:00:32.149 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:00:32.214 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T11:00:32.214 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T11:00:32.214 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000200172 s, 2.6 MB/s 2024-04-25T11:00:32.215 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:00:32.275 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-04-25T11:00:32.332 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 166895 Links: 1 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 11:00:23.200382706 +0000 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 10:57:38.533872178 +0000 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 10:57:38.533872178 +0000 2024-04-25T11:00:32.333 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-04-25T11:00:32.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:00:32.397 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T11:00:32.397 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T11:00:32.397 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000203377 s, 2.5 MB/s 2024-04-25T11:00:32.398 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:00:32.457 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:00:32.457 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:00:32.457 INFO:tasks.ceph:Generating config... 2024-04-25T11:00:32.458 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-25T11:00:32.458 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-04-25T11:00:32.458 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2024-04-25T11:00:32.458 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2024-04-25T11:00:32.459 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-04-25T11:00:32.459 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T11:00:32.459 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T11:00:32.459 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T11:00:32.459 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T11:00:32.459 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T11:00:32.459 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T11:00:32.460 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2024-04-25T11:00:32.460 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T11:00:32.460 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T11:00:32.460 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T11:00:32.460 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T11:00:32.460 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T11:00:32.460 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T11:00:32.461 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T11:00:32.461 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T11:00:32.462 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T11:00:32.629 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T11:00:32.632 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T11:00:32.672 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:00:32.739 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.155'), ('mon.b', '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]'), ('mon.c', '[v2:172.21.15.155:3302,v1:172.21.15.155:6791]')] 2024-04-25T11:00:32.739 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790],[v2:172.21.15.155:3302,v1:172.21.15.155:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T11:00:32.740 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:32.740 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:00:32.797 DEBUG:teuthology.orchestra.run.smithi155:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.155 --addv b '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]' --addv c '[v2:172.21.15.155:3302,v1:172.21.15.155:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:monmaptool: generated fsid f023c376-0581-4e8c-aaca-f3f02914116c 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:epoch 0 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:fsid f023c376-0581-4e8c-aaca-f3f02914116c 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:last_changed 2024-04-25T11:00:32.883721+0000 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:created 2024-04-25T11:00:32.883721+0000 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:min_mon_release 0 (unknown) 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:election_strategy: 1 2024-04-25T11:00:32.883 INFO:teuthology.orchestra.run.smithi155.stdout:0: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.a 2024-04-25T11:00:32.884 INFO:teuthology.orchestra.run.smithi155.stdout:1: [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] mon.b 2024-04-25T11:00:32.884 INFO:teuthology.orchestra.run.smithi155.stdout:2: [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] mon.c 2024-04-25T11:00:32.884 INFO:teuthology.orchestra.run.smithi155.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T11:00:32.885 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:00:32.902 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f023c376-0581-4e8c-aaca-f3f02914116c... 2024-04-25T11:00:32.903 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:00:32.986 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-04-25T11:00:32.987 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:00:32.988 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-04-25T11:00:32.989 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790],[v2:172.21.15.155:3302,v1:172.21.15.155:6791]" 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on pool no app = False 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: ms bind msgr2 = False 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore warn on legacy statfs = False 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore warn on no per pool omap = False 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn min per osd = 0 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = f023c376-0581-4e8c-aaca-f3f02914116c 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-04-25T11:00:32.990 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.991 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2024-04-25T11:00:32.992 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:00:32.993 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:00:32.994 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-04-25T11:00:32.995 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = False 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-04-25T11:00:32.996 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-04-25T11:00:32.997 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-04-25T11:00:32.998 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T11:00:32.998 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T11:00:33.090 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T11:00:33.090 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:33.090 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T11:00:33.108 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:33.108 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T11:00:33.166 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:00:33.166 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:33.166 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:00:33.166 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:00:33.242 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:33.242 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:33.300 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T11:00:33.300 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T11:00:33.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:00:33.390 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:00:33.392 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:00:33.440 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:00:33.442 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T11:00:33.442 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T11:00:33.442 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T11:00:33.442 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:00:33.442 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T11:00:33.469 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:00:33.469 INFO:tasks.ceph:role: osd.0 2024-04-25T11:00:33.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:00:33.469 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:00:33.544 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:00:33.971 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T11:00:33.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T11:00:33.972 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T11:00:34.136 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T11:00:34.171 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T11:00:34.239 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:00:34.240 INFO:tasks.ceph:role: osd.1 2024-04-25T11:00:34.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:00:34.240 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:00:34.314 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:00:34.315 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:00:34.315 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:00:34.315 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:00:34.734 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T11:00:34.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T11:00:34.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T11:00:34.844 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T11:00:34.875 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T11:00:34.943 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:00:34.943 INFO:tasks.ceph:role: osd.2 2024-04-25T11:00:34.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:00:34.943 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:00:35.018 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:00:35.018 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:00:35.018 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:00:35.018 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:00:35.018 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:00:35.018 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:00:35.019 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:00:35.019 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:00:35.019 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:00:35.019 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:00:35.472 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T11:00:35.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T11:00:35.474 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T11:00:35.626 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T11:00:35.660 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T11:00:35.727 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:00:35.727 INFO:tasks.ceph:role: osd.3 2024-04-25T11:00:35.727 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:00:35.728 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:00:35.801 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:00:35.802 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:00:36.243 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T11:00:36.244 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T11:00:36.244 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T11:00:36.420 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T11:00:36.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:36.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:36.547+0000 7f47257eb380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T11:00:36.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:36.547+0000 7f47257eb380 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T11:00:36.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:36.547+0000 7f47257eb380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:00:36.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:36.547+0000 7f47257eb380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:00:36.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:36.547+0000 7f47257eb380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:00:36.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:36.547+0000 7f47257eb380 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T11:00:39.391 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T11:00:39.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:39.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:39.513+0000 7f5685c9b380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T11:00:39.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:39.513+0000 7f5685c9b380 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T11:00:39.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:39.513+0000 7f5685c9b380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:00:39.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:39.513+0000 7f5685c9b380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:00:39.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:39.513+0000 7f5685c9b380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:00:39.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:39.514+0000 7f5685c9b380 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T11:00:42.355 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T11:00:42.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:42.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:42.469+0000 7fd5cc266380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T11:00:42.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:42.469+0000 7fd5cc266380 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T11:00:42.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:42.469+0000 7fd5cc266380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:00:42.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:42.469+0000 7fd5cc266380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:00:42.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:42.470+0000 7fd5cc266380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:00:42.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:42.470+0000 7fd5cc266380 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T11:00:45.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T11:00:45.354 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:45.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:45.443+0000 7f17541b2380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T11:00:45.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:45.443+0000 7f17541b2380 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T11:00:45.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:45.443+0000 7f17541b2380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:00:45.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:45.443+0000 7f17541b2380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:00:45.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:45.443+0000 7f17541b2380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:00:45.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:45.444+0000 7f17541b2380 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T11:00:48.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T11:00:48.319 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T11:00:48.319 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:48.319 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T11:00:48.389 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:48.389 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T11:00:48.458 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:48.458 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T11:00:48.527 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:48.527 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T11:00:48.595 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:48.595 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T11:00:48.663 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:48.663 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T11:00:48.730 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T11:00:48.730 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDRNypmcy1KFxAASlZY/X9uDqD6XO/rT3rhPg== 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.y] 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDRNypmsFFJGhAAXXiby+gAgvlIivhQVkK1UQ== 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDUNypmJVKeIBAADZAaVKC0gW7Pqt1HIxEiXw== 2024-04-25T11:00:48.796 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2024-04-25T11:00:48.797 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDXNypmEOOcHhAAlVVePnwpv0iekkXSK8lPFg== 2024-04-25T11:00:48.797 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2024-04-25T11:00:48.797 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDaNypm4LQRHBAAZVtBcF4Xnz6dBJ3i5CnuxA== 2024-04-25T11:00:48.797 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2024-04-25T11:00:48.797 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDdNypmFM1/GhAAbTVUXVe8fUdNT1N03MCkoA== 2024-04-25T11:00:48.798 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:00:48.887 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:00:48.935 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:00:48.984 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:00:49.032 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:00:49.080 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:00:49.128 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T11:00:49.128 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T11:00:49.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:00:49.247 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T11:00:49.273 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T11:00:49.340 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:00:49.433 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T11:00:49.460 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T11:00:49.527 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:00:49.621 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T11:00:49.647 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:00:49.705 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T11:00:49.705 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T11:00:49.705 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T11:00:49.748 INFO:tasks.ceph.mon.a:Started 2024-04-25T11:00:49.748 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T11:00:49.748 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T11:00:49.750 INFO:tasks.ceph.mon.b:Started 2024-04-25T11:00:49.750 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T11:00:49.750 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T11:00:49.752 INFO:tasks.ceph.mon.c:Started 2024-04-25T11:00:49.752 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T11:00:49.752 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T11:00:49.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T11:00:49.754 INFO:tasks.ceph.mgr.x:Started 2024-04-25T11:00:49.755 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T11:00:49.755 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T11:00:49.757 INFO:tasks.ceph.mgr.y:Started 2024-04-25T11:00:49.757 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T11:00:49.757 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T11:00:49.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.894+0000 7fd4e1193700 1 Processor -- start 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.894+0000 7fd4e1193700 1 -- start start 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4e1193700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 0x7fd4dc122410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4e1193700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 0x7fd4dc12a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4e1193700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc069b40 con 0x7fd4dc122900 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4e1193700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc069020 con 0x7fd4dc122030 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4e1193700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc068890 con 0x7fd4dc11f2e0 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4da59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd4dc122900 0x7fd4dc11ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:50528/0 (socket says 172.21.15.155:50528) 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4da59c700 1 -- 172.21.15.155:0/2609743961 learned_addr learned my addr 172.21.15.155:0/2609743961 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4db59e700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 0x7fd4dc12a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4dad9d700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 0x7fd4dc122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4db59e700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 msgr2=0x7fd4dc12a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4db59e700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 msgr2=0x7fd4dc12a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4dad9d700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 msgr2=0x7fd4dc122410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4db59e700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 0x7fd4dc12a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T11:00:49.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4dad9d700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 msgr2=0x7fd4dc122410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:00:49.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4dad9d700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 0x7fd4dc122410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T11:00:49.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4dad9d700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 0x7fd4dc122410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:00:49.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:49.895+0000 7fd4db59e700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 0x7fd4dc12a3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:00:50.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.085+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342777126 0 0) 0x7fd4dc069b40 con 0x7fd4dc122900 2024-04-25T11:00:50.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c40033f0 con 0x7fd4dc122900 2024-04-25T11:00:50.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 <== mon.0 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3425305382 0 0) 0x7fd4c40033f0 con 0x7fd4dc122900 2024-04-25T11:00:50.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 msgr2=0x7fd4dc12a3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:00:50.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4dc11f2e0 0x7fd4dc12a3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 msgr2=0x7fd4dc122410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc122030 0x7fd4dc122410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc069880 con 0x7fd4dc122900 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 <== mon.0 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4012276641 0 0) 0x7fd4cc001470 con 0x7fd4dc122900 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 >> v1:172.21.15.155:6789/0 conn(0x7fd4dc122900 legacy=0x7fd4dc11ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4003910 0x7fd4c4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4004690 0x7fd4c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4005000 0x7fd4c400fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc068890 con 0x7fd4c4003910 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c4004440 con 0x7fd4c4004690 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40103e0 con 0x7fd4c4005000 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4db59e700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4005000 0x7fd4c400fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.086+0000 7fd4da59c700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4004690 0x7fd4c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4da59c700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4005000 msgr2=0x7fd4c400fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4da59c700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4005000 0x7fd4c400fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4da59c700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4003910 msgr2=0x7fd4c4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4da59c700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4003910 0x7fd4c4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:00:50.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4da59c700 1 -- 172.21.15.155:0/2609743961 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40033f0 con 0x7fd4c4004690 2024-04-25T11:00:50.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4da59c700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4004690 0x7fd4c4004b10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fd4cc00c190 tx=0x7fd4cc001e50).ready entity=mon.1 client_cookie=b6a0b06084b725ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc007a20 con 0x7fd4c4004690 2024-04-25T11:00:50.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00ca80 con 0x7fd4c4004690 2024-04-25T11:00:50.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.087+0000 7fd4d9d9b700 1 -- 172.21.15.155:0/2609743961 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00ce20 con 0x7fd4c4004690 2024-04-25T11:00:50.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 -- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4004690 msgr2=0x7fd4c4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4004690 0x7fd4c4004b10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fd4cc00c190 tx=0x7fd4cc001e50).stop 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 -- 172.21.15.155:0/2609743961 shutdown_connections 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4005000 0x7fd4c400fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4003910 0x7fd4c4003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 --2- 172.21.15.155:0/2609743961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4004690 0x7fd4c4004b10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 -- 172.21.15.155:0/2609743961 >> 172.21.15.155:0/2609743961 conn(0x7fd4dc0956f0 msgr2=0x7fd4dc095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 -- 172.21.15.155:0/2609743961 shutdown_connections 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.088+0000 7fd4e1193700 1 -- 172.21.15.155:0/2609743961 wait complete. 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 Processor -- start 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 -- start start 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003910 0x7fd4dc130420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4004690 0x7fd4dc130910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4005000 0x7fd4dc1116f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc114d20 con 0x7fd4c4005000 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc1143a0 con 0x7fd4c4003910 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4e1193700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc114f50 con 0x7fd4c4004690 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4db59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4005000 0x7fd4dc1116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4db59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4005000 0x7fd4dc1116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:48978/0 (socket says 172.21.15.155:48978) 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4da59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4004690 0x7fd4dc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.090+0000 7fd4db59e700 1 -- 172.21.15.155:0/351902562 learned_addr learned my addr 172.21.15.155:0/351902562 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4dad9d700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003910 0x7fd4dc130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4da59c700 1 -- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003910 msgr2=0x7fd4dc130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4da59c700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003910 0x7fd4dc130420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4da59c700 1 -- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4005000 msgr2=0x7fd4dc1116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4da59c700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4005000 0x7fd4dc1116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4da59c700 1 -- 172.21.15.155:0/351902562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc1131e0 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4da59c700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4004690 0x7fd4dc130910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd4cc012000 tx=0x7fd4cc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0048c0 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc004a20 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc004de0 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc130e00 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc1310e0 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fd4cc0138c0 con 0x7fd4c4004690 2024-04-25T11:00:50.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.091+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd4cc0207d0 con 0x7fd4c4004690 2024-04-25T11:00:50.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.093+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc114f50 con 0x7fd4c4004690 2024-04-25T11:00:50.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.096+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fd4cc03fe50 con 0x7fd4c4004690 2024-04-25T11:00:50.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.169+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd4dc114730 con 0x7fd4c4004690 2024-04-25T11:00:50.174 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.174+0000 7fd4c37fe700 1 -- 172.21.15.155:0/351902562 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd4cc02e020 con 0x7fd4c4004690 2024-04-25T11:00:50.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.175+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4004690 msgr2=0x7fd4dc130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.175+0000 7fd4e1193700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4004690 0x7fd4dc130910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd4cc012000 tx=0x7fd4cc015000).stop 2024-04-25T11:00:50.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 shutdown_connections 2024-04-25T11:00:50.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c4005000 0x7fd4dc1116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003910 0x7fd4dc130420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 --2- 172.21.15.155:0/351902562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd4c4004690 0x7fd4dc130910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 >> 172.21.15.155:0/351902562 conn(0x7fd4dc0956f0 msgr2=0x7fd4dc093ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 shutdown_connections 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.176+0000 7fd4e1193700 1 -- 172.21.15.155:0/351902562 wait complete. 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi155.stderr:adjusted tunables profile to default 2024-04-25T11:00:50.189 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T11:00:50.189 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T11:00:50.189 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:50.189 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T11:00:50.215 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:50.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T11:00:50.248 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:50.247+0000 7fe7b9e1e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:00:50.248 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:50.247+0000 7fae9f760680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:00:50.274 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:50.274 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T11:00:50.320 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:50.320+0000 7fae9f760680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:00:50.321 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:50.320+0000 7fe7b9e1e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:00:50.341 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:00:50.341 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T11:00:50.395 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:50.395+0000 7fae9f760680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:00:50.396 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:50.396+0000 7fe7b9e1e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:00:50.410 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new a1a66333-2acf-451b-b7d0-2dfa2e55600f 0 2024-04-25T11:00:50.549 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:50.548+0000 7fae9f760680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:00:50.549 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:50.549+0000 7fe7b9e1e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:00:50.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.566+0000 7fca1729b700 1 Processor -- start 2024-04-25T11:00:50.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.566+0000 7fca1729b700 1 -- start start 2024-04-25T11:00:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca1729b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 0x7fca10122940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca1729b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca10090fb0 0x7fca1012a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca1729b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca10067120 con 0x7fca10090460 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca1729b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1006a3a0 con 0x7fca10090fb0 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca1729b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca10122110 con 0x7fca10122560 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 0x7fca10122940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 0x7fca10122940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:54944/0 (socket says 172.21.15.155:54944) 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 -- 172.21.15.155:0/1435298895 learned_addr learned my addr 172.21.15.155:0/1435298895 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15838700 1 --2- 172.21.15.155:0/1435298895 >> v2:172.21.15.155:3300/0 conn(0x7fca10090fb0 0x7fca1012a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 -- 172.21.15.155:0/1435298895 >> v2:172.21.15.155:3300/0 conn(0x7fca10090fb0 msgr2=0x7fca1012a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 --2- 172.21.15.155:0/1435298895 >> v2:172.21.15.155:3300/0 conn(0x7fca10090fb0 0x7fca1012a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 -- 172.21.15.155:0/1435298895 >> v1:172.21.15.155:6789/0 conn(0x7fca10090460 legacy=0x7fca100908a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca07fff700 1 -- 172.21.15.155:0/1435298895 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781368441 0 0) 0x7fca00001110 con 0x7fca10090460 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15037700 1 -- 172.21.15.155:0/1435298895 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1005e300 con 0x7fca10122560 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.567+0000 7fca15838700 1 --2- 172.21.15.155:0/1435298895 >> v2:172.21.15.155:3300/0 conn(0x7fca10090fb0 0x7fca1012a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca15037700 1 --2- 172.21.15.155:0/1435298895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 0x7fca10122940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fca0c00bae0 tx=0x7fca0c00b300).ready entity=mon.1 client_cookie=3f3804cbf0a8151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca07fff700 1 -- 172.21.15.155:0/1435298895 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00da30 con 0x7fca10122560 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca07fff700 1 -- 172.21.15.155:0/1435298895 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c00db90 con 0x7fca10122560 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca07fff700 1 -- 172.21.15.155:0/1435298895 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c005880 con 0x7fca10122560 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca1729b700 1 -- 172.21.15.155:0/1435298895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 msgr2=0x7fca10122940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca1729b700 1 --2- 172.21.15.155:0/1435298895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 0x7fca10122940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fca0c00bae0 tx=0x7fca0c00b300).stop 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca1729b700 1 -- 172.21.15.155:0/1435298895 shutdown_connections 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca1729b700 1 --2- 172.21.15.155:0/1435298895 >> v2:172.21.15.155:3300/0 conn(0x7fca10090fb0 0x7fca1012a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca1729b700 1 --2- 172.21.15.155:0/1435298895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10122560 0x7fca10122940 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.568+0000 7fca1729b700 1 -- 172.21.15.155:0/1435298895 >> 172.21.15.155:0/1435298895 conn(0x7fca10095700 msgr2=0x7fca100915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.569+0000 7fca1729b700 1 -- 172.21.15.155:0/1435298895 shutdown_connections 2024-04-25T11:00:50.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.569+0000 7fca1729b700 1 -- 172.21.15.155:0/1435298895 wait complete. 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 Processor -- start 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 -- start start 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca10090460 0x7fca1011dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 0x7fca1011e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fca10122560 0x7fca1011e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca10121290 con 0x7fca10090460 2024-04-25T11:00:50.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca10084090 con 0x7fca10090fb0 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca1729b700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca10084930 con 0x7fca10122560 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca14836700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 0x7fca1011e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.570+0000 7fca14836700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 0x7fca1011e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:54952/0 (socket says 172.21.15.155:54952) 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 -- 172.21.15.155:0/3154692642 learned_addr learned my addr 172.21.15.155:0/3154692642 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca15838700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fca10122560 0x7fca1011e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca15037700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca10090460 0x7fca1011dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 -- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fca10122560 msgr2=0x7fca1011e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fca10122560 0x7fca1011e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 -- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca10090460 msgr2=0x7fca1011dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca10090460 0x7fca1011dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 -- 172.21.15.155:0/3154692642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08009a20 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca15838700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fca10122560 0x7fca1011e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca14836700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 0x7fca1011e4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fca0000c440 tx=0x7fca00007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000cea0 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0000f470 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c00b5a0 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000f810 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.571+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1008dd80 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.572+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fca0001c440 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.572+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fca00012750 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.572+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca10068120 con 0x7fca10090fb0 2024-04-25T11:00:50.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.575+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fca00017070 con 0x7fca10090fb0 2024-04-25T11:00:50.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.655+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a1a66333-2acf-451b-b7d0-2dfa2e55600f", "id": 0} v 0) v1 -- 0x7fca10121e50 con 0x7fca10090fb0 2024-04-25T11:00:50.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.666+0000 7fca05f76700 1 -- 172.21.15.155:0/3154692642 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1a66333-2acf-451b-b7d0-2dfa2e55600f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fca10067120 con 0x7fca10090fb0 2024-04-25T11:00:50.667 INFO:teuthology.orchestra.run.smithi155.stdout:0 2024-04-25T11:00:50.668 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 msgr2=0x7fca1011e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.668 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 0x7fca1011e4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fca0000c440 tx=0x7fca00007a60).stop 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 shutdown_connections 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fca10122560 0x7fca1011e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca10090460 0x7fca1011dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 --2- 172.21.15.155:0/3154692642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca10090fb0 0x7fca1011e4a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 >> 172.21.15.155:0/3154692642 conn(0x7fca10095700 msgr2=0x7fca1008e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 shutdown_connections 2024-04-25T11:00:50.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.667+0000 7fca1729b700 1 -- 172.21.15.155:0/3154692642 wait complete. 2024-04-25T11:00:50.680 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 7051096c-4e97-4ace-9814-455da6c8a06b 1 2024-04-25T11:00:50.718 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:50.717+0000 7fae9f760680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:00:50.720 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:50.719+0000 7fe7b9e1e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:00:50.806 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.805+0000 7f7395504700 1 Processor -- start 2024-04-25T11:00:50.806 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.805+0000 7f7395504700 1 -- start start 2024-04-25T11:00:50.808 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f7395504700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 0x7f7390122770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.808 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f7395504700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122c60 0x7f7390090460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.808 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f7395504700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73901221d0 con 0x7f7390090950 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f7395504700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7390067ba0 con 0x7f7390122390 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f7395504700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f739005e300 con 0x7f7390122c60 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f738effd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 0x7f7390122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f738e7fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122c60 0x7f7390090460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f738effd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 0x7f7390122770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:54962/0 (socket says 172.21.15.155:54962) 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f738effd700 1 -- 172.21.15.155:0/3171138916 learned_addr learned my addr 172.21.15.155:0/3171138916 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.806+0000 7f738f7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7390090950 0x7f73901293f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:50544/0 (socket says 172.21.15.155:50544) 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738effd700 1 -- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122c60 msgr2=0x7f7390090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738effd700 1 --2- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122c60 0x7f7390090460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738dffb700 1 -- 172.21.15.155:0/3171138916 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443486280 0 0) 0x7f73901221d0 con 0x7f7390090950 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738effd700 1 -- 172.21.15.155:0/3171138916 >> v1:172.21.15.155:6789/0 conn(0x7f7390090950 legacy=0x7f73901293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738effd700 1 -- 172.21.15.155:0/3171138916 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7390067430 con 0x7f7390122390 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738e7fc700 1 --2- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122c60 0x7f7390090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738effd700 1 --2- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 0x7f7390122770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7380009840 tx=0x7f7380009060).ready entity=mon.1 client_cookie=1e9a592ed44135a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738dffb700 1 -- 172.21.15.155:0/3171138916 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380005880 con 0x7f7390122390 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738dffb700 1 -- 172.21.15.155:0/3171138916 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73800059e0 con 0x7f7390122390 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.807+0000 7f738dffb700 1 -- 172.21.15.155:0/3171138916 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380005d60 con 0x7f7390122390 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 -- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 msgr2=0x7f7390122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 --2- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 0x7f7390122770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7380009840 tx=0x7f7380009060).stop 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 -- 172.21.15.155:0/3171138916 shutdown_connections 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 --2- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122390 0x7f7390122770 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 --2- 172.21.15.155:0/3171138916 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122c60 0x7f7390090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 -- 172.21.15.155:0/3171138916 >> 172.21.15.155:0/3171138916 conn(0x7f7390095700 msgr2=0x7f73900915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 -- 172.21.15.155:0/3171138916 shutdown_connections 2024-04-25T11:00:50.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.808+0000 7f7395504700 1 -- 172.21.15.155:0/3171138916 wait complete. 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.809+0000 7f7395504700 1 Processor -- start 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.809+0000 7f7395504700 1 -- start start 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f7395504700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7390090950 0x7f739011e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f7395504700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122390 0x7f739011ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f7395504700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122c60 0x7f739011f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f7395504700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7390125150 con 0x7f7390090950 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f7395504700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73901236f0 con 0x7f7390122c60 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f7395504700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73901238b0 con 0x7f7390122390 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738e7fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122390 0x7f739011ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738e7fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122390 0x7f739011ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52722/0 (socket says 172.21.15.155:52722) 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738e7fc700 1 -- 172.21.15.155:0/3428397523 learned_addr learned my addr 172.21.15.155:0/3428397523 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:50.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122c60 0x7f739011f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738effd700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7390090950 0x7f739011e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 -- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122390 msgr2=0x7f739011ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122390 0x7f739011ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 -- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7390090950 msgr2=0x7f739011e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7390090950 0x7f739011e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 -- 172.21.15.155:0/3428397523 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840092a0 con 0x7f7390122c60 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.810+0000 7f738f7fe700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122c60 0x7f739011f1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f738800c7e0 tx=0x7f7388007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388005c70 con 0x7f7390122c60 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7388005df0 con 0x7f7390122c60 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7380009300 con 0x7f7390122c60 2024-04-25T11:00:50.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388007360 con 0x7f7390122c60 2024-04-25T11:00:50.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73901248b0 con 0x7f7390122c60 2024-04-25T11:00:50.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f738801d440 con 0x7f7390122c60 2024-04-25T11:00:50.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.811+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f7388013410 con 0x7f7390122c60 2024-04-25T11:00:50.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.812+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7390047540 con 0x7f7390122c60 2024-04-25T11:00:50.816 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.814+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f738801d8d0 con 0x7f7390122c60 2024-04-25T11:00:50.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.894+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7051096c-4e97-4ace-9814-455da6c8a06b", "id": 1} v 0) v1 -- 0x7f739006a2c0 con 0x7f7390122c60 2024-04-25T11:00:50.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.902+0000 7f73777fe700 1 -- 172.21.15.155:0/3428397523 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7051096c-4e97-4ace-9814-455da6c8a06b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f738801da70 con 0x7f7390122c60 2024-04-25T11:00:50.909 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T11:00:50.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122c60 msgr2=0x7f739011f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122c60 0x7f739011f1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f738800c7e0 tx=0x7f7388007eb0).stop 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 shutdown_connections 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7390122c60 0x7f739011f1b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7390090950 0x7f739011e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 --2- 172.21.15.155:0/3428397523 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7390122390 0x7f739011ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 >> 172.21.15.155:0/3428397523 conn(0x7f7390095700 msgr2=0x7f73900683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 shutdown_connections 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:50.904+0000 7f7395504700 1 -- 172.21.15.155:0/3428397523 wait complete. 2024-04-25T11:00:50.916 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new faf1e6c0-e94a-4bf3-830d-67564885faaa 2 2024-04-25T11:00:51.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.075+0000 7f7d30442700 1 Processor -- start 2024-04-25T11:00:51.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.075+0000 7f7d30442700 1 -- start start 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d30442700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d28122030 0x7f7d28122410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d30442700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d28122900 0x7f7d2811ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d30442700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d2811f0c0 0x7f7d2811f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d30442700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d28069020 con 0x7f7d28122030 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d30442700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2811fa60 con 0x7f7d28122900 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d30442700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d28069b40 con 0x7f7d2811f0c0 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d2d9dd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d28122900 0x7f7d2811ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d2e1de700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d28122030 0x7f7d28122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d2d9dd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d28122900 0x7f7d2811ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:54988/0 (socket says 172.21.15.155:54988) 2024-04-25T11:00:51.079 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d2d9dd700 1 -- 172.21.15.155:0/1109420746 learned_addr learned my addr 172.21.15.155:0/1109420746 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.076+0000 7f7d2e9df700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d2811f0c0 0x7f7d2811f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e1de700 1 -- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d2811f0c0 msgr2=0x7f7d2811f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e1de700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d2811f0c0 0x7f7d2811f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e1de700 1 -- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d28122900 msgr2=0x7f7d2811ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e1de700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d28122900 0x7f7d2811ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e1de700 1 -- 172.21.15.155:0/1109420746 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28068890 con 0x7f7d28122030 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e9df700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d2811f0c0 0x7f7d2811f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2e1de700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122030 0x7f7d28122410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f7d24009760 tx=0x7f7d24009060).ready entity=mon.0 client_cookie=eb6f822b1ddb6510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2d1dc700 1 -- 172.21.15.155:0/1109420746 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240108a0 con 0x7f7d28122030 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2d1dc700 1 -- 172.21.15.155:0/1109420746 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24010a00 con 0x7f7d28122030 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.077+0000 7f7d2d1dc700 1 -- 172.21.15.155:0/1109420746 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24010b70 con 0x7f7d28122030 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 -- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122030 msgr2=0x7f7d28122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122030 0x7f7d28122410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f7d24009760 tx=0x7f7d24009060).stop 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 -- 172.21.15.155:0/1109420746 shutdown_connections 2024-04-25T11:00:51.080 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d2811f0c0 0x7f7d2811f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122030 0x7f7d28122410 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 --2- 172.21.15.155:0/1109420746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d28122900 0x7f7d2811ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 -- 172.21.15.155:0/1109420746 >> 172.21.15.155:0/1109420746 conn(0x7f7d280956f0 msgr2=0x7f7d28095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 -- 172.21.15.155:0/1109420746 shutdown_connections 2024-04-25T11:00:51.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.078+0000 7f7d30442700 1 -- 172.21.15.155:0/1109420746 wait complete. 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.079+0000 7f7d30442700 1 Processor -- start 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.079+0000 7f7d30442700 1 -- start start 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d30442700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2811f0c0 0x7f7d28111d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d30442700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d28122030 0x7f7d281121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d30442700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 0x7f7d281126e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d30442700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d281139f0 con 0x7f7d28122900 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d30442700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d28115680 con 0x7f7d2811f0c0 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d30442700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2811e870 con 0x7f7d28122030 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 0x7f7d281126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e1de700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2811f0c0 0x7f7d28111d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 0x7f7d281126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49012/0 (socket says 172.21.15.155:49012) 2024-04-25T11:00:51.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 -- 172.21.15.155:0/3645465573 learned_addr learned my addr 172.21.15.155:0/3645465573 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2d9dd700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d28122030 0x7f7d281121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 -- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d28122030 msgr2=0x7f7d281121f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d28122030 0x7f7d281121f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 -- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2811f0c0 msgr2=0x7f7d28111d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2811f0c0 0x7f7d28111d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e9df700 1 -- 172.21.15.155:0/3645465573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2000ba20 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.080+0000 7f7d2e1de700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2811f0c0 0x7f7d28111d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d2e9df700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 0x7f7d281126e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7d20008400 tx=0x7f7d2000b780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20010040 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20015480 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d240092a0 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20008ea0 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d28114a40 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f7d20015c70 con 0x7f7d28122900 2024-04-25T11:00:51.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.081+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f7d2001f440 con 0x7f7d28122900 2024-04-25T11:00:51.086 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.082+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28047540 con 0x7f7d28122900 2024-04-25T11:00:51.086 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.084+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f7d2001a070 con 0x7f7d28122900 2024-04-25T11:00:51.179 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:51.178+0000 7fae9f760680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:00:51.182 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:51.182+0000 7fe7b9e1e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:00:51.184 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.184+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "faf1e6c0-e94a-4bf3-830d-67564885faaa", "id": 2} v 0) v1 -- 0x7f7d28093f70 con 0x7f7d28122900 2024-04-25T11:00:51.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.192+0000 7f7d1effd700 1 -- 172.21.15.155:0/3645465573 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "faf1e6c0-e94a-4bf3-830d-67564885faaa", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d2001f650 con 0x7f7d28122900 2024-04-25T11:00:51.193 INFO:teuthology.orchestra.run.smithi155.stdout:2 2024-04-25T11:00:51.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.193+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 msgr2=0x7f7d281126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.193+0000 7f7d30442700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 0x7f7d281126e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7d20008400 tx=0x7f7d2000b780).stop 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 shutdown_connections 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28122900 0x7f7d281126e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2811f0c0 0x7f7d28111d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 --2- 172.21.15.155:0/3645465573 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d28122030 0x7f7d281121f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 >> 172.21.15.155:0/3645465573 conn(0x7f7d280956f0 msgr2=0x7f7d28121330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 shutdown_connections 2024-04-25T11:00:51.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.194+0000 7f7d30442700 1 -- 172.21.15.155:0/3645465573 wait complete. 2024-04-25T11:00:51.206 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 4e269fdb-6854-450b-9956-6ab00516289e 3 2024-04-25T11:00:51.290 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:51.290+0000 7fe7b9e1e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:00:51.308 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:51.308+0000 7fae9f760680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 Processor -- start 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 -- start start 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e50090460 0x7f5e50065760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50065c50 0x7f5e50066050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e50067be0 con 0x7f5e5008e050 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e50067920 con 0x7f5e50090460 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e56393700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50066540 con 0x7f5e50065c50 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e4ffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e50090460 0x7f5e50065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.343 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.340+0000 7f5e54930700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5e5008e050 0x7f5e5008e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:60728/0 (socket says 172.21.15.155:60728) 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e54930700 1 -- 172.21.15.155:0/3799233858 learned_addr learned my addr 172.21.15.155:0/3799233858 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4f7fe700 1 --2- 172.21.15.155:0/3799233858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50065c50 0x7f5e50066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4f7fe700 1 -- 172.21.15.155:0/3799233858 >> v2:172.21.15.155:3300/0 conn(0x7f5e50090460 msgr2=0x7f5e50065760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4f7fe700 1 --2- 172.21.15.155:0/3799233858 >> v2:172.21.15.155:3300/0 conn(0x7f5e50090460 0x7f5e50065760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4f7fe700 1 -- 172.21.15.155:0/3799233858 >> v1:172.21.15.155:6789/0 conn(0x7f5e5008e050 legacy=0x7f5e5008e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4f7fe700 1 -- 172.21.15.155:0/3799233858 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500912a0 con 0x7f5e50065c50 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4ffff700 1 --2- 172.21.15.155:0/3799233858 >> v2:172.21.15.155:3300/0 conn(0x7f5e50090460 0x7f5e50065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4f7fe700 1 --2- 172.21.15.155:0/3799233858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50065c50 0x7f5e50066050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f5e44009760 tx=0x7f5e44009060).ready entity=mon.2 client_cookie=ee25d4d4b2dc0957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4effd700 1 -- 172.21.15.155:0/3799233858 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400f510 con 0x7f5e50065c50 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4effd700 1 -- 172.21.15.155:0/3799233858 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4400f670 con 0x7f5e50065c50 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.341+0000 7f5e4effd700 1 -- 172.21.15.155:0/3799233858 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400f9f0 con 0x7f5e50065c50 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 -- 172.21.15.155:0/3799233858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50065c50 msgr2=0x7f5e50066050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 --2- 172.21.15.155:0/3799233858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50065c50 0x7f5e50066050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f5e44009760 tx=0x7f5e44009060).stop 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 -- 172.21.15.155:0/3799233858 shutdown_connections 2024-04-25T11:00:51.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 --2- 172.21.15.155:0/3799233858 >> v2:172.21.15.155:3300/0 conn(0x7f5e50090460 0x7f5e50065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 --2- 172.21.15.155:0/3799233858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50065c50 0x7f5e50066050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 -- 172.21.15.155:0/3799233858 >> 172.21.15.155:0/3799233858 conn(0x7f5e50095700 msgr2=0x7f5e500915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 -- 172.21.15.155:0/3799233858 shutdown_connections 2024-04-25T11:00:51.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.342+0000 7f5e56393700 1 -- 172.21.15.155:0/3799233858 wait complete. 2024-04-25T11:00:51.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 Processor -- start 2024-04-25T11:00:51.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 -- start start 2024-04-25T11:00:51.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 0x7f5e50119e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e5008e050 0x7f5e5011a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50090460 0x7f5e5011a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50119bd0 con 0x7f5e50065c50 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5011bb10 con 0x7f5e5008e050 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.343+0000 7f5e56393700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5011bc80 con 0x7f5e50090460 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 0x7f5e50119e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 0x7f5e50119e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39666/0 (socket says 172.21.15.155:39666) 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 -- 172.21.15.155:0/1316051385 learned_addr learned my addr 172.21.15.155:0/1316051385 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4f7fe700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e5008e050 0x7f5e5011a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 -- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50090460 msgr2=0x7f5e5011a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50090460 0x7f5e5011a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 -- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e5008e050 msgr2=0x7f5e5011a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e5008e050 0x7f5e5011a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 -- 172.21.15.155:0/1316051385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4000a3f0 con 0x7f5e50065c50 2024-04-25T11:00:51.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4f7fe700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e5008e050 0x7f5e5011a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4ffff700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 0x7f5e50119e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5e4000e970 tx=0x7f5e4000a170).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40005880 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400059e0 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e440092a0 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40005d60 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.344+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5008cd90 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.345+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e50047540 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.345+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f5e40006490 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.345+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f5e40023420 con 0x7f5e50065c50 2024-04-25T11:00:51.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.348+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f5e40015030 con 0x7f5e50065c50 2024-04-25T11:00:51.359 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:51.358+0000 7fe7b9e1e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:00:51.377 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:51.377+0000 7fae9f760680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:00:51.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.428+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4e269fdb-6854-450b-9956-6ab00516289e", "id": 3} v 0) v1 -- 0x7f5e50092460 con 0x7f5e50065c50 2024-04-25T11:00:51.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.436+0000 7f5e4cff9700 1 -- 172.21.15.155:0/1316051385 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4e269fdb-6854-450b-9956-6ab00516289e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5e4000b040 con 0x7f5e50065c50 2024-04-25T11:00:51.437 INFO:teuthology.orchestra.run.smithi155.stdout:3 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 msgr2=0x7f5e50119e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 0x7f5e50119e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5e4000e970 tx=0x7f5e4000a170).stop 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 shutdown_connections 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e50090460 0x7f5e5011a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e50065c50 0x7f5e50119e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 --2- 172.21.15.155:0/1316051385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e5008e050 0x7f5e5011a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 >> 172.21.15.155:0/1316051385 conn(0x7f5e50095700 msgr2=0x7f5e50092e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 shutdown_connections 2024-04-25T11:00:51.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.437+0000 7f5e56393700 1 -- 172.21.15.155:0/1316051385 wait complete. 2024-04-25T11:00:51.449 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T11:00:51.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T11:00:51.451 INFO:tasks.ceph.osd.0:Started 2024-04-25T11:00:51.452 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T11:00:51.452 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T11:00:51.453 INFO:tasks.ceph.osd.1:Started 2024-04-25T11:00:51.454 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T11:00:51.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T11:00:51.455 INFO:tasks.ceph.osd.2:Started 2024-04-25T11:00:51.456 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T11:00:51.456 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T11:00:51.458 INFO:tasks.ceph.osd.3:Started 2024-04-25T11:00:51.458 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:51.533 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:51.531+0000 7fe7b9e1e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:00:51.547 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:51.546+0000 7fae9f760680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:00:51.619 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:51.618+0000 7fe7b9e1e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:00:51.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.618+0000 7fe559e20700 1 Processor -- start 2024-04-25T11:00:51.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.618+0000 7fe559e20700 1 -- start start 2024-04-25T11:00:51.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe559e20700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe554090460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe559e20700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe554090d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe559e20700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe554066cc0 con 0x7fe554129cb0 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe559e20700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe554066f10 con 0x7fe554090950 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe559e20700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe554091220 con 0x7fe55405e140 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe553fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe554129cb0 0x7fe55412c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:60764/0 (socket says 172.21.15.155:60764) 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5537fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe554090460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe553fff700 1 -- 172.21.15.155:0/719113282 learned_addr learned my addr 172.21.15.155:0/719113282 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe552ffd700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe554090d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5527fc700 1 -- 172.21.15.155:0/719113282 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153618313 0 0) 0x7fe554066cc0 con 0x7fe554129cb0 2024-04-25T11:00:51.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5527fc700 1 -- 172.21.15.155:0/719113282 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe53c0033f0 con 0x7fe554129cb0 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5537fe700 1 -- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 msgr2=0x7fe554090d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5537fe700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe554090d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5537fe700 1 -- 172.21.15.155:0/719113282 >> v1:172.21.15.155:6789/0 conn(0x7fe554129cb0 legacy=0x7fe55412c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5537fe700 1 -- 172.21.15.155:0/719113282 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe554065e80 con 0x7fe55405e140 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe552ffd700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe554090d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.619+0000 7fe5537fe700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe554090460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe544009760 tx=0x7fe544006f40).ready entity=mon.2 client_cookie=c7ccf3f9a21a900e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe5527fc700 1 -- 172.21.15.155:0/719113282 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544005880 con 0x7fe55405e140 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe5527fc700 1 -- 172.21.15.155:0/719113282 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5440059e0 con 0x7fe55405e140 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe5527fc700 1 -- 172.21.15.155:0/719113282 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544005d60 con 0x7fe55405e140 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 -- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 msgr2=0x7fe554090460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe554090460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe544009760 tx=0x7fe544006f40).stop 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 -- 172.21.15.155:0/719113282 shutdown_connections 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe554090460 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 --2- 172.21.15.155:0/719113282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe554090d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.623 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 -- 172.21.15.155:0/719113282 >> 172.21.15.155:0/719113282 conn(0x7fe554095700 msgr2=0x7fe5540915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.624 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 -- 172.21.15.155:0/719113282 shutdown_connections 2024-04-25T11:00:51.624 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.620+0000 7fe559e20700 1 -- 172.21.15.155:0/719113282 wait complete. 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.621+0000 7fe559e20700 1 Processor -- start 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.621+0000 7fe559e20700 1 -- start start 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe559e20700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe55411a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe559e20700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe55411a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe559e20700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe554129cb0 0x7fe55411ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe559e20700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55411d6a0 con 0x7fe554129cb0 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe559e20700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55411da70 con 0x7fe554090950 2024-04-25T11:00:51.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe559e20700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe55411bd90 con 0x7fe55405e140 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe55411a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe55411a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50000/0 (socket says 172.21.15.155:50000) 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 -- 172.21.15.155:0/1254741558 learned_addr learned my addr 172.21.15.155:0/1254741558 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe553fff700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe554129cb0 0x7fe55411ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe552ffd700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe55411a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 -- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 msgr2=0x7fe55411a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe55411a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 -- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe554129cb0 msgr2=0x7fe55411ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe554129cb0 0x7fe55411ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 -- 172.21.15.155:0/1254741558 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544009250 con 0x7fe55405e140 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.622+0000 7fe5537fe700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe55411a180 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe5540913b0 tx=0x7fe544015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544006950 con 0x7fe55405e140 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe544006690 con 0x7fe55405e140 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55411b050 con 0x7fe55405e140 2024-04-25T11:00:51.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54400f740 con 0x7fe55405e140 2024-04-25T11:00:51.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55411d080 con 0x7fe55405e140 2024-04-25T11:00:51.628 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fae9f760680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:00:51.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fe54401d440 con 0x7fe55405e140 2024-04-25T11:00:51.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.623+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fe54400fdd0 con 0x7fe55405e140 2024-04-25T11:00:51.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.624+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55411d210 con 0x7fe55405e140 2024-04-25T11:00:51.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.626+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fe54402c060 con 0x7fe55405e140 2024-04-25T11:00:51.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.692+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe554047540 con 0x7fe55405e140 2024-04-25T11:00:51.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.693+0000 7fe53bfff700 1 -- 172.21.15.155:0/1254741558 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7fe544035090 con 0x7fe55405e140 2024-04-25T11:00:51.694 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:00:51.694 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":6,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:00:51.430090+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 msgr2=0x7fe55411a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe55411a180 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe5540913b0 tx=0x7fe544015000).stop 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 shutdown_connections 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe554129cb0 0x7fe55411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe55405e140 0x7fe55411a180 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 --2- 172.21.15.155:0/1254741558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe554090950 0x7fe55411a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 >> 172.21.15.155:0/1254741558 conn(0x7fe554095700 msgr2=0x7fe554068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 shutdown_connections 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:51.694+0000 7fe559e20700 1 -- 172.21.15.155:0/1254741558 wait complete. 2024-04-25T11:00:51.707 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T11:00:51.707 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T11:00:51.830 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:00:51.830+0000 7f66ac47e380 -1 Falling back to public interface 2024-04-25T11:00:52.007 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.007+0000 7fe7b9e1e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:00:52.008 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.007+0000 7fae9f760680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:00:52.084 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:00:52.083+0000 7f38ae5f0380 -1 Falling back to public interface 2024-04-25T11:00:52.089 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:00:52.088+0000 7fa004abe380 -1 Falling back to public interface 2024-04-25T11:00:52.095 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:00:52.094+0000 7f1e85660380 -1 Falling back to public interface 2024-04-25T11:00:52.095 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.095+0000 7fae9f760680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:00:52.097 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.097+0000 7fe7b9e1e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:00:52.109 DEBUG:teuthology.orchestra.run.smithi155:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:52.220 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 Processor -- start 2024-04-25T11:00:52.220 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 -- start start 2024-04-25T11:00:52.220 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 0x7f9e2011c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:52.220 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011ca50 0x7f9e2008d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e20064960 con 0x7f9e2008d840 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2008eef0 con 0x7f9e2011c180 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e25b79700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2008e8f0 con 0x7f9e2011ca50 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e1f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 0x7f9e2011c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e1ffff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f9e2008d840 0x7f9e20129630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:60776/0 (socket says 172.21.15.155:60776) 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e1ffff700 1 -- 172.21.15.155:0/1403538971 learned_addr learned my addr 172.21.15.155:0/1403538971 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e1f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 0x7f9e2011c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34764/0 (socket says 172.21.15.155:34764) 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.218+0000 7f9e1effd700 1 --2- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011ca50 0x7f9e2008d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1e7fc700 1 -- 172.21.15.155:0/1403538971 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745439055 0 0) 0x7f9e20064960 con 0x7f9e2008d840 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011ca50 msgr2=0x7f9e2008d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1f7fe700 1 --2- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011ca50 0x7f9e2008d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/1403538971 >> v1:172.21.15.155:6789/0 conn(0x7f9e2008d840 legacy=0x7f9e20129630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/1403538971 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e20062ba0 con 0x7f9e2011c180 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1f7fe700 1 --2- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 0x7f9e2011c560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f9e10009790 tx=0x7f9e10009060).ready entity=mon.1 client_cookie=1b3dcb407ee2b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.221 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1e7fc700 1 -- 172.21.15.155:0/1403538971 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000f510 con 0x7f9e2011c180 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1e7fc700 1 -- 172.21.15.155:0/1403538971 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000f670 con 0x7f9e2011c180 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.219+0000 7f9e1e7fc700 1 -- 172.21.15.155:0/1403538971 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000f9f0 con 0x7f9e2011c180 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 -- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 msgr2=0x7f9e2011c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 --2- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 0x7f9e2011c560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f9e10009790 tx=0x7f9e10009060).stop 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 -- 172.21.15.155:0/1403538971 shutdown_connections 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 --2- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2011c180 0x7f9e2011c560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 --2- 172.21.15.155:0/1403538971 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011ca50 0x7f9e2008d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 -- 172.21.15.155:0/1403538971 >> 172.21.15.155:0/1403538971 conn(0x7f9e20094cd0 msgr2=0x7f9e200950e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 -- 172.21.15.155:0/1403538971 shutdown_connections 2024-04-25T11:00:52.222 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.220+0000 7f9e25b79700 1 -- 172.21.15.155:0/1403538971 wait complete. 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 Processor -- start 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 -- start start 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 0x7f9e2011e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011c180 0x7f9e2011e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e2011ca50 0x7f9e201219d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e201315e0 con 0x7f9e2011ca50 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2011f180 con 0x7f9e2008d840 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e25b79700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20120da0 con 0x7f9e2011c180 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e1f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 0x7f9e2011e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e1ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e2011ca50 0x7f9e201219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e1f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 0x7f9e2011e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34778/0 (socket says 172.21.15.155:34778) 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/3396646928 learned_addr learned my addr 172.21.15.155:0/3396646928 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:52.225 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.221+0000 7f9e1effd700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011c180 0x7f9e2011e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011c180 msgr2=0x7f9e2011e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1f7fe700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011c180 0x7f9e2011e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e2011ca50 msgr2=0x7f9e201219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1f7fe700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e2011ca50 0x7f9e201219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1f7fe700 1 -- 172.21.15.155:0/3396646928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100092a0 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1ffff700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e2011ca50 0x7f9e201219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e1f7fe700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 0x7f9e2011e220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9e1000f610 tx=0x7f9e10017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000fe80 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000f7e0 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1001b640 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2011fa20 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2011ec00 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f9e1001bc50 con 0x7f9e2008d840 2024-04-25T11:00:52.226 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.222+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f9e10024b40 con 0x7f9e2008d840 2024-04-25T11:00:52.227 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.223+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2005a310 con 0x7f9e2008d840 2024-04-25T11:00:52.227 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.226+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f9e10029060 con 0x7f9e2008d840 2024-04-25T11:00:52.320 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.318+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e20046ce0 con 0x7f9e2008d840 2024-04-25T11:00:52.320 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.319+0000 7f9e07fff700 1 -- 172.21.15.155:0/3396646928 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7f9e1001f070 con 0x7f9e2008d840 2024-04-25T11:00:52.320 INFO:teuthology.misc.health.smithi155.stdout: 2024-04-25T11:00:52.321 INFO:teuthology.misc.health.smithi155.stdout:{"epoch":6,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:00:51.430090+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:52.322 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 msgr2=0x7f9e2011e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 0x7f9e2011e220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9e1000f610 tx=0x7f9e10017000).stop 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 shutdown_connections 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e2011ca50 0x7f9e201219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e2008d840 0x7f9e2011e220 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 --2- 172.21.15.155:0/3396646928 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9e2011c180 0x7f9e2011e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 >> 172.21.15.155:0/3396646928 conn(0x7f9e20094cd0 msgr2=0x7f9e20119f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.323 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 shutdown_connections 2024-04-25T11:00:52.324 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:52.320+0000 7f9e25b79700 1 -- 172.21.15.155:0/3396646928 wait complete. 2024-04-25T11:00:52.334 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-25T11:00:52.438 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.438+0000 7fe7b9e1e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:00:52.457 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.456+0000 7fae9f760680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:00:52.519 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.518+0000 7fe7b9e1e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:00:52.531 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.530+0000 7fae9f760680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:00:52.668 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.667+0000 7fe7b9e1e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:00:52.675 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.675+0000 7fae9f760680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:00:52.740 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.739+0000 7fe7b9e1e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:00:52.744 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.744+0000 7fae9f760680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:00:52.986 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:52.985+0000 7fe7b9e1e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:00:52.989 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:52.989+0000 7fae9f760680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:00:53.135 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:53.135+0000 7fe7b9e1e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:00:53.136 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:53.135+0000 7fae9f760680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:00:53.298 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:53.297+0000 7fe7b9e1e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:00:53.298 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:53.297+0000 7fae9f760680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:00:53.370 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:53.369+0000 7fae9f760680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:00:53.370 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:00:53.369+0000 7fe7b9e1e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:00:54.398 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:54.397+0000 7fae7679c700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T11:00:54.398 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:00:54.397+0000 7fae7679c700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T11:00:55.334 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:00:55.333+0000 7f38ae5f0380 -1 osd.3 0 log_to_monitors {default=true} 2024-04-25T11:00:55.354 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:00:55.353+0000 7f66ac47e380 -1 osd.0 0 log_to_monitors {default=true} 2024-04-25T11:00:55.558 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:00:55.557+0000 7fa004abe380 -1 osd.1 0 log_to_monitors {default=true} 2024-04-25T11:00:55.611 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:00:55.607+0000 7f1e85660380 -1 osd.2 0 log_to_monitors {default=true} 2024-04-25T11:00:56.407 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:00:56.406+0000 7f38a4f91700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:00:56.407 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:00:56.406+0000 7f66a2e1f700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:00:56.412 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:00:56.411+0000 7f669f59b700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:00:56.412 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:00:56.411+0000 7f38a170d700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:00:57.444 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:00:57.444+0000 7f9ffb45f700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:00:57.445 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:00:57.444+0000 7f1e7c001700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:00:57.448 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:00:57.447+0000 7f9ff7bdb700 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:00:57.448 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:00:57.448+0000 7f1e7877d700 -1 osd.2 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:00:58.737 DEBUG:teuthology.orchestra.run.smithi155:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:58.858 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.855+0000 7f2313257700 1 Processor -- start 2024-04-25T11:00:58.858 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.855+0000 7f2313257700 1 -- start start 2024-04-25T11:00:58.858 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2313257700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f230c125790 0x7f230c125b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:58.858 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2313257700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 0x7f230c091a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:58.858 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2313257700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c0923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:58.858 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2313257700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230c0928a0 con 0x7f230c125790 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2313257700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c097a70 con 0x7f230c091f00 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2313257700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f230c097d20 con 0x7f230c126060 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f230bfff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 0x7f230c091a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c0923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2310ff3700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f230c125790 0x7f230c125b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f230bfff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 0x7f230c091a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50082/0 (socket says 172.21.15.155:50082) 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c0923b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34862/0 (socket says 172.21.15.155:34862) 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f2310ff3700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f230c125790 0x7f230c125b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39748/0 (socket says 172.21.15.155:39748) 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f230bfff700 1 -- 172.21.15.155:0/2070431422 learned_addr learned my addr 172.21.15.155:0/2070431422 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 -- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 msgr2=0x7f230c091a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 --2- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 0x7f230c091a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.859 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 -- 172.21.15.155:0/2070431422 >> v2:172.21.15.155:3300/0 conn(0x7f230c125790 msgr2=0x7f230c125b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 --2- 172.21.15.155:0/2070431422 >> v2:172.21.15.155:3300/0 conn(0x7f230c125790 0x7f230c125b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 -- 172.21.15.155:0/2070431422 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c097fe0 con 0x7f230c091f00 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f230bfff700 1 --2- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 0x7f230c091a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f23117f4700 1 --2- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c0923b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2304009e90 tx=0x7f2304009780).ready entity=mon.1 client_cookie=43767fcea3a8e8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.856+0000 7f230b7fe700 1 -- 172.21.15.155:0/2070431422 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400e070 con 0x7f230c091f00 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f230b7fe700 1 -- 172.21.15.155:0/2070431422 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400ba70 con 0x7f230c091f00 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f230b7fe700 1 -- 172.21.15.155:0/2070431422 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400bdf0 con 0x7f230c091f00 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 -- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 msgr2=0x7f230c0923b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 --2- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c0923b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2304009e90 tx=0x7f2304009780).stop 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 -- 172.21.15.155:0/2070431422 shutdown_connections 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 --2- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c0923b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 --2- 172.21.15.155:0/2070431422 >> v2:172.21.15.155:3300/0 conn(0x7f230c125790 0x7f230c125b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.860 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 --2- 172.21.15.155:0/2070431422 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c126060 0x7f230c091a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.861 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 -- 172.21.15.155:0/2070431422 >> 172.21.15.155:0/2070431422 conn(0x7f230c098d20 msgr2=0x7f230c095200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:58.861 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 -- 172.21.15.155:0/2070431422 shutdown_connections 2024-04-25T11:00:58.861 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.857+0000 7f2313257700 1 -- 172.21.15.155:0/2070431422 wait complete. 2024-04-25T11:00:58.863 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.858+0000 7f2313257700 1 Processor -- start 2024-04-25T11:00:58.863 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.858+0000 7f2313257700 1 -- start start 2024-04-25T11:00:58.863 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:58.863 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c125790 0x7f230c134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f230c126060 0x7f230c113be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c117790 con 0x7f230c126060 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c116920 con 0x7f230c091f00 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f230c117100 con 0x7f230c125790 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c134a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34872/0 (socket says 172.21.15.155:34872) 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 -- 172.21.15.155:0/2987949182 learned_addr learned my addr 172.21.15.155:0/2987949182 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f230bfff700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c125790 0x7f230c134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f23117f4700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f230c126060 0x7f230c113be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 -- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c125790 msgr2=0x7f230c134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c125790 0x7f230c134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 -- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f230c126060 msgr2=0x7f230c113be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f230c126060 0x7f230c113be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:58.864 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 -- 172.21.15.155:0/2987949182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009370 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f230bfff700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c125790 0x7f230c134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2310ff3700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c134a80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f230000e930 tx=0x7f2300009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300009510 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300012600 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2304009a20 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300012980 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.859+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c135460 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.860+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f230001b400 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.860+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c05a310 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.861+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2461+0+0 (secure 0 0 0) 0x7f2300055400 con 0x7f230c091f00 2024-04-25T11:00:58.865 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:58.865+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f2300035f20 con 0x7f230c091f00 2024-04-25T11:00:59.005 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.005+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f230c046ce0 con 0x7f230c091f00 2024-04-25T11:00:59.006 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.005+0000 7f23097fa700 1 -- 172.21.15.155:0/2987949182 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6553 (secure 0 0 0) 0x7f2300039710 con 0x7f230c091f00 2024-04-25T11:00:59.006 INFO:teuthology.misc.health.smithi155.stdout: 2024-04-25T11:00:59.006 INFO:teuthology.misc.health.smithi155.stdout:{"epoch":10,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:00:58.405672+0000","last_up_change":"2024-04-25T11:00:58.405672+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-04-25T11:00:57.443026+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":43900}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":43900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":43900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":43900}]},"public_addr":"172.21.15.155:6800/43900","cluster_addr":"172.21.15.155:6801/43900","heartbeat_back_addr":"172.21.15.155:6803/43900","heartbeat_front_addr":"172.21.15.155:6802/43900","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 >> v1:172.21.15.155:6816/43298 conn(0x7f22f0041b80 legacy=0x7f22f0044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 msgr2=0x7f230c134a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c134a80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f230000e930 tx=0x7f2300009060).stop 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 shutdown_connections 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f230c126060 0x7f230c113be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f230c091f00 0x7f230c134a80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 --2- 172.21.15.155:0/2987949182 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f230c125790 0x7f230c134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.008 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 >> 172.21.15.155:0/2987949182 conn(0x7f230c098d20 msgr2=0x7f230c124210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.009 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 shutdown_connections 2024-04-25T11:00:59.009 INFO:teuthology.misc.health.smithi155.stderr:2024-04-25T11:00:59.007+0000 7f2313257700 1 -- 172.21.15.155:0/2987949182 wait complete. 2024-04-25T11:00:59.018 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-25T11:00:59.018 INFO:tasks.ceph:Creating RBD pool 2024-04-25T11:00:59.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.132+0000 7f8e50ab8700 1 Processor -- start 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.132+0000 7f8e50ab8700 1 -- start start 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.132+0000 7f8e50ab8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 0x7f8e4c065760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.132+0000 7f8e50ab8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c065c50 0x7f8e4c066050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e50ab8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e4c08e050 0x7f8e4c08e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e50ab8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c067be0 con 0x7f8e4c08e050 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e50ab8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c067920 con 0x7f8e4c065c50 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e50ab8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c066540 con 0x7f8e4c090460 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4a59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 0x7f8e4c065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4a59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 0x7f8e4c065760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50110/0 (socket says 172.21.15.155:50110) 2024-04-25T11:00:59.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e4c08e050 0x7f8e4c08e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4a59c700 1 -- 172.21.15.155:0/1733271558 learned_addr learned my addr 172.21.15.155:0/1733271558 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c065c50 0x7f8e4c066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 -- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 msgr2=0x7f8e4c065760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 0x7f8e4c065760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 -- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c065c50 msgr2=0x7f8e4c066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c065c50 0x7f8e4c066050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 -- 172.21.15.155:0/1733271558 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0912a0 con 0x7f8e4c08e050 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4a59c700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 0x7f8e4c065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e49d9b700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c065c50 0x7f8e4c066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4ad9d700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c08e500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8e4400d0d0 tx=0x7f8e4400b990).ready entity=mon.0 client_cookie=f2cc4e533b2729d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4959a700 1 -- 172.21.15.155:0/1733271558 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44010070 con 0x7f8e4c08e050 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4959a700 1 -- 172.21.15.155:0/1733271558 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400dc90 con 0x7f8e4c08e050 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.133+0000 7f8e4959a700 1 -- 172.21.15.155:0/1733271558 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44015680 con 0x7f8e4c08e050 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 -- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 msgr2=0x7f8e4c08e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c08e500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8e4400d0d0 tx=0x7f8e4400b990).stop 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 -- 172.21.15.155:0/1733271558 shutdown_connections 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c08e500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c090460 0x7f8e4c065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/1733271558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c065c50 0x7f8e4c066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 -- 172.21.15.155:0/1733271558 >> 172.21.15.155:0/1733271558 conn(0x7f8e4c095700 msgr2=0x7f8e4c0915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 -- 172.21.15.155:0/1733271558 shutdown_connections 2024-04-25T11:00:59.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.134+0000 7f8e50ab8700 1 -- 172.21.15.155:0/1733271558 wait complete. 2024-04-25T11:00:59.139 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 Processor -- start 2024-04-25T11:00:59.139 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 -- start start 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c065c50 0x7f8e4c08d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c090460 0x7f8e4c130b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c08ba30 con 0x7f8e4c08e050 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c08dc80 con 0x7f8e4c090460 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e50ab8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c08ddf0 con 0x7f8e4c065c50 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e4a59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c065c50 0x7f8e4c08d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.135+0000 7f8e4ad9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c090460 0x7f8e4c130b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4a59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c065c50 0x7f8e4c08d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50122/0 (socket says 172.21.15.155:50122) 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c130640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39760/0 (socket says 172.21.15.155:39760) 2024-04-25T11:00:59.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4a59c700 1 -- 172.21.15.155:0/3125883025 learned_addr learned my addr 172.21.15.155:0/3125883025 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4ad9d700 1 -- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c065c50 msgr2=0x7f8e4c08d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4ad9d700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c065c50 0x7f8e4c08d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4ad9d700 1 -- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 msgr2=0x7f8e4c130640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4ad9d700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c130640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4ad9d700 1 -- 172.21.15.155:0/3125883025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0092a0 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e49d9b700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e4ad9d700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c090460 0x7f8e4c130b30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f8e44017000 tx=0x7f8e4400b990).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400b240 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400b3a0 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40009300 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440096b0 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.136+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4400bc30 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.137+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f8e4401c070 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.137+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c08c2a0 con 0x7f8e4c090460 2024-04-25T11:00:59.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.137+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2461+0+0 (secure 0 0 0) 0x7f8e44056650 con 0x7f8e4c090460 2024-04-25T11:00:59.142 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.141+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f8e44023020 con 0x7f8e4c090460 2024-04-25T11:00:59.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.279+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f8e4c047540 con 0x7f8e4c090460 2024-04-25T11:00:59.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.415+0000 7f8e3affd700 1 -- 172.21.15.155:0/3125883025 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v11) v1 ==== 109+0+0 (secure 0 0 0) 0x7f8e4403a9f0 con 0x7f8e4c090460 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.417+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 >> v1:172.21.15.155:6816/43298 conn(0x7f8e2c041b80 legacy=0x7f8e2c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.417+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c090460 msgr2=0x7f8e4c130b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.417+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c090460 0x7f8e4c130b30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f8e44017000 tx=0x7f8e4400b990).stop 2024-04-25T11:00:59.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 shutdown_connections 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e4c090460 0x7f8e4c130b30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e4c065c50 0x7f8e4c08d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 --2- 172.21.15.155:0/3125883025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e4c08e050 0x7f8e4c130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 >> 172.21.15.155:0/3125883025 conn(0x7f8e4c095700 msgr2=0x7f8e4c068780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 shutdown_connections 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.418+0000 7f8e50ab8700 1 -- 172.21.15.155:0/3125883025 wait complete. 2024-04-25T11:00:59.419 INFO:teuthology.orchestra.run.smithi155.stderr:pool 'rbd' created 2024-04-25T11:00:59.431 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.547+0000 7f8b96914700 1 Processor -- start 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.547+0000 7f8b96914700 1 -- start start 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b96914700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 0x7f8b90066610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b96914700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 0x7f8b90125330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b96914700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8b90125820 0x7f8b90125cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b96914700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b901261a0 con 0x7f8b90125820 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b96914700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90068730 con 0x7f8b90066b00 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b96914700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90066010 con 0x7f8b90066230 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 0x7f8b90066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 0x7f8b90125330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 0x7f8b90066610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50134/0 (socket says 172.21.15.155:50134) 2024-04-25T11:00:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8ffff700 1 -- 172.21.15.155:0/2787244346 learned_addr learned my addr 172.21.15.155:0/2787244346 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 0x7f8b90125330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34890/0 (socket says 172.21.15.155:34890) 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b94eb1700 1 --2- 172.21.15.155:0/2787244346 >> v2:172.21.15.155:3300/0 conn(0x7f8b90125820 0x7f8b90125cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 -- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 msgr2=0x7f8b90066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 --2- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 0x7f8b90066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 -- 172.21.15.155:0/2787244346 >> v2:172.21.15.155:3300/0 conn(0x7f8b90125820 msgr2=0x7f8b90125cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 --2- 172.21.15.155:0/2787244346 >> v2:172.21.15.155:3300/0 conn(0x7f8b90125820 0x7f8b90125cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 -- 172.21.15.155:0/2787244346 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b90066f40 con 0x7f8b90066b00 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8ffff700 1 --2- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 0x7f8b90066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b94eb1700 1 --2- 172.21.15.155:0/2787244346 >> v2:172.21.15.155:3300/0 conn(0x7f8b90125820 0x7f8b90125cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8f7fe700 1 --2- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 0x7f8b90125330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8b80009760 tx=0x7f8b80009060).ready entity=mon.1 client_cookie=36c6e3589bd4bc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8effd700 1 -- 172.21.15.155:0/2787244346 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000d4a0 con 0x7f8b90066b00 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8effd700 1 -- 172.21.15.155:0/2787244346 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8000d600 con 0x7f8b90066b00 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.548+0000 7f8b8effd700 1 -- 172.21.15.155:0/2787244346 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000d980 con 0x7f8b90066b00 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 -- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 msgr2=0x7f8b90125330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 --2- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 0x7f8b90125330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8b80009760 tx=0x7f8b80009060).stop 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 -- 172.21.15.155:0/2787244346 shutdown_connections 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 --2- 172.21.15.155:0/2787244346 >> v2:172.21.15.155:3300/0 conn(0x7f8b90125820 0x7f8b90125cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 --2- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90066230 0x7f8b90066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 --2- 172.21.15.155:0/2787244346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066b00 0x7f8b90125330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 -- 172.21.15.155:0/2787244346 >> 172.21.15.155:0/2787244346 conn(0x7f8b90095760 msgr2=0x7f8b90095b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 -- 172.21.15.155:0/2787244346 shutdown_connections 2024-04-25T11:00:59.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.549+0000 7f8b96914700 1 -- 172.21.15.155:0/2787244346 wait complete. 2024-04-25T11:00:59.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 Processor -- start 2024-04-25T11:00:59.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 -- start start 2024-04-25T11:00:59.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066230 0x7f8b90117520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b90066b00 0x7f8b90117a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90125820 0x7f8b90117f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9011ab90 con 0x7f8b90066b00 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9011a210 con 0x7f8b90066230 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b96914700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90122d80 con 0x7f8b90125820 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066230 0x7f8b90117520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b94eb1700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90125820 0x7f8b90117f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066230 0x7f8b90117520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34904/0 (socket says 172.21.15.155:34904) 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b8ffff700 1 -- 172.21.15.155:0/753055345 learned_addr learned my addr 172.21.15.155:0/753055345 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.550+0000 7f8b8f7fe700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b90066b00 0x7f8b90117a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b94eb1700 1 -- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066230 msgr2=0x7f8b90117520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b94eb1700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066230 0x7f8b90117520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b94eb1700 1 -- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b90066b00 msgr2=0x7f8b90117a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b94eb1700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b90066b00 0x7f8b90117a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:00:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b94eb1700 1 -- 172.21.15.155:0/753055345 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b84009370 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b8f7fe700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b90066b00 0x7f8b90117a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b94eb1700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90125820 0x7f8b90117f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f8b88009f10 tx=0x7f8b8800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88009c70 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8800f040 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b88009a20 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800cd80 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.551+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800092a0 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.552+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f8b88013610 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.552+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9011a4a0 con 0x7f8b90125820 2024-04-25T11:00:59.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.552+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2827+0+0 (secure 0 0 0) 0x7f8b880559d0 con 0x7f8b90125820 2024-04-25T11:00:59.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.556+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f8b9011a4a0 con 0x7f8b90125820 2024-04-25T11:00:59.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:00:59.699+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b9011a4a0 con 0x7f8b90125820 2024-04-25T11:01:00.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.422+0000 7f8b8cff9700 1 -- 172.21.15.155:0/753055345 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v12) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8b9011a4a0 con 0x7f8b90125820 2024-04-25T11:01:00.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 >> v1:172.21.15.155:6816/43298 conn(0x7f8b7c041b80 legacy=0x7f8b7c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90125820 msgr2=0x7f8b90117f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90125820 0x7f8b90117f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f8b88009f10 tx=0x7f8b8800b040).stop 2024-04-25T11:01:00.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 shutdown_connections 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8b90125820 0x7f8b90117f00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b90066230 0x7f8b90117520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 --2- 172.21.15.155:0/753055345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b90066b00 0x7f8b90117a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.424+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 >> 172.21.15.155:0/753055345 conn(0x7f8b90095760 msgr2=0x7f8b90124260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.425+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 shutdown_connections 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.425+0000 7f8b96914700 1 -- 172.21.15.155:0/753055345 wait complete. 2024-04-25T11:01:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-25T11:01:00.441 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T11:01:00.442 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:01:00.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:01:00.442 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:01:00.443 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T11:01:00.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 Processor -- start 2024-04-25T11:01:00.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 -- start start 2024-04-25T11:01:00.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d900626c0 0x7f7d90125460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 0x7f7d90125950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900676e0 con 0x7f7d90125e40 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d90067870 con 0x7f7d900626c0 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.562+0000 7f7d985a9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90066660 con 0x7f7d90067bc0 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95b44700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 0x7f7d90125950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95b44700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 0x7f7d90125950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50148/0 (socket says 172.21.15.155:50148) 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d96345700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d900626c0 0x7f7d90125460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95b44700 1 -- 172.21.15.155:0/1267814133 learned_addr learned my addr 172.21.15.155:0/1267814133 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d96345700 1 -- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 msgr2=0x7f7d90125950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d96345700 1 --2- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 0x7f7d90125950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d96345700 1 -- 172.21.15.155:0/1267814133 >> v1:172.21.15.155:6789/0 conn(0x7f7d90125e40 legacy=0x7f7d90065f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d96345700 1 -- 172.21.15.155:0/1267814133 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900667d0 con 0x7f7d900626c0 2024-04-25T11:01:00.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95b44700 1 --2- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 0x7f7d90125950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95343700 1 -- 172.21.15.155:0/1267814133 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276960080 0 0) 0x7f7d900676e0 con 0x7f7d90125e40 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d96345700 1 --2- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d900626c0 0x7f7d90125460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7d8c009760 tx=0x7f7d8c009060).ready entity=mon.0 client_cookie=4cba1608e7124ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95343700 1 -- 172.21.15.155:0/1267814133 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00f510 con 0x7f7d900626c0 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95343700 1 -- 172.21.15.155:0/1267814133 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c00f670 con 0x7f7d900626c0 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.563+0000 7f7d95343700 1 -- 172.21.15.155:0/1267814133 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00f9f0 con 0x7f7d900626c0 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 -- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d900626c0 msgr2=0x7f7d90125460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 --2- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d900626c0 0x7f7d90125460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7d8c009760 tx=0x7f7d8c009060).stop 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 -- 172.21.15.155:0/1267814133 shutdown_connections 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 --2- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d900626c0 0x7f7d90125460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 --2- 172.21.15.155:0/1267814133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90067bc0 0x7f7d90125950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 -- 172.21.15.155:0/1267814133 >> 172.21.15.155:0/1267814133 conn(0x7f7d900956f0 msgr2=0x7f7d90095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 -- 172.21.15.155:0/1267814133 shutdown_connections 2024-04-25T11:01:00.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.564+0000 7f7d985a9700 1 -- 172.21.15.155:0/1267814133 wait complete. 2024-04-25T11:01:00.568 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 Processor -- start 2024-04-25T11:01:00.568 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 -- start start 2024-04-25T11:01:00.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d900626c0 0x7f7d90134d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 0x7f7d90135200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90125e40 0x7f7d90113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90117110 con 0x7f7d90067bc0 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90116790 con 0x7f7d900626c0 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d985a9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90117340 con 0x7f7d90125e40 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 0x7f7d90135200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d96b46700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90125e40 0x7f7d90113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 0x7f7d90135200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39808/0 (socket says 172.21.15.155:39808) 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 -- 172.21.15.155:0/342833854 learned_addr learned my addr 172.21.15.155:0/342833854 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d96345700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d900626c0 0x7f7d90134d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 -- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90125e40 msgr2=0x7f7d90113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90125e40 0x7f7d90113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 -- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d900626c0 msgr2=0x7f7d90134d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d900626c0 0x7f7d90134d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d95b44700 1 -- 172.21.15.155:0/342833854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80009370 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d96345700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d900626c0 0x7f7d90134d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.565+0000 7f7d96b46700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90125e40 0x7f7d90113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.566+0000 7f7d95b44700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 0x7f7d90135200 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7d8000ea20 tx=0x7f7d800090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.566+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80009650 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.566+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000ee60 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.566+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8c0092a0 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.566+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000ca00 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.566+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d901356f0 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.567+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d90116b20 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.567+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f7d8001b400 con 0x7f7d90067bc0 2024-04-25T11:01:00.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.568+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f7d80055bd0 con 0x7f7d90067bc0 2024-04-25T11:01:00.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.571+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f7d80036610 con 0x7f7d90067bc0 2024-04-25T11:01:00.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.742+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7d90135970 con 0x7f7d90067bc0 2024-04-25T11:01:00.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.744+0000 7f7d86ffd700 1 -- 172.21.15.155:0/342833854 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+97488 (secure 0 0 0) 0x7f7d80029070 con 0x7f7d90067bc0 2024-04-25T11:01:00.746 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:00.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 >> v1:172.21.15.155:6816/43298 conn(0x7f7d74041b80 legacy=0x7f7d74044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 msgr2=0x7f7d90135200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 0x7f7d90135200 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7d8000ea20 tx=0x7f7d800090f0).stop 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 shutdown_connections 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7d90125e40 0x7f7d90113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d900626c0 0x7f7d90134d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 --2- 172.21.15.155:0/342833854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90067bc0 0x7f7d90135200 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 >> 172.21.15.155:0/342833854 conn(0x7f7d900956f0 msgr2=0x7f7d90091a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 shutdown_connections 2024-04-25T11:01:00.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.746+0000 7f7d985a9700 1 -- 172.21.15.155:0/342833854 wait complete. 2024-04-25T11:01:00.758 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":4,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6816","nonce":43298}]},"active_addr":"172.21.15.155:6816/43298","active_change":"2024-04-25T11:00:53.384392+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":966131479}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":3077341019}]}]} 2024-04-25T11:01:00.763 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:01:00.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:01:00.763 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:01:00.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.876+0000 7f89189fa700 1 Processor -- start 2024-04-25T11:01:00.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.876+0000 7f89189fa700 1 -- start start 2024-04-25T11:01:00.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f89189fa700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914090f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f89189fa700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 0x7f891412d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f89189fa700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8914066cc0 con 0x7f891405e140 2024-04-25T11:01:00.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f89189fa700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8914066f10 con 0x7f891412acc0 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f89189fa700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8914065e80 con 0x7f8914090b70 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f8912d9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 0x7f891412d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f8911d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914090f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f8912d9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 0x7f891412d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39814/0 (socket says 172.21.15.155:39814) 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f8912d9d700 1 -- 172.21.15.155:0/3007315909 learned_addr learned my addr 172.21.15.155:0/3007315909 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f891259c700 1 --1- 172.21.15.155:0/3007315909 >> v1:172.21.15.155:6789/0 conn(0x7f891405e140 0x7f8914090460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:60830/0 (socket says 172.21.15.155:60830) 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f8911d9b700 1 -- 172.21.15.155:0/3007315909 >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 msgr2=0x7f891412d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.877+0000 7f8911d9b700 1 --2- 172.21.15.155:0/3007315909 >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 0x7f891412d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f8911d9b700 1 -- 172.21.15.155:0/3007315909 >> v1:172.21.15.155:6789/0 conn(0x7f891405e140 legacy=0x7f8914090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f8911d9b700 1 -- 172.21.15.155:0/3007315909 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8914066360 con 0x7f8914090b70 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f891159a700 1 -- 172.21.15.155:0/3007315909 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250934895 0 0) 0x7f8914066cc0 con 0x7f891405e140 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f8912d9d700 1 --2- 172.21.15.155:0/3007315909 >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 0x7f891412d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:00.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f8911d9b700 1 --2- 172.21.15.155:0/3007315909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914090f50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8904009760 tx=0x7f8904009060).ready entity=mon.1 client_cookie=b18572521a15c4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f891159a700 1 -- 172.21.15.155:0/3007315909 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89040090d0 con 0x7f8914090b70 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f891159a700 1 -- 172.21.15.155:0/3007315909 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890400f510 con 0x7f8914090b70 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f891159a700 1 -- 172.21.15.155:0/3007315909 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400f890 con 0x7f8914090b70 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 -- 172.21.15.155:0/3007315909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 msgr2=0x7f8914090f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 --2- 172.21.15.155:0/3007315909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914090f50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8904009760 tx=0x7f8904009060).stop 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 -- 172.21.15.155:0/3007315909 shutdown_connections 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 --2- 172.21.15.155:0/3007315909 >> v2:172.21.15.155:3300/0 conn(0x7f891412acc0 0x7f891412d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 --2- 172.21.15.155:0/3007315909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914090f50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 -- 172.21.15.155:0/3007315909 >> 172.21.15.155:0/3007315909 conn(0x7f8914095700 msgr2=0x7f89140915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 -- 172.21.15.155:0/3007315909 shutdown_connections 2024-04-25T11:01:00.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.878+0000 7f89189fa700 1 -- 172.21.15.155:0/3007315909 wait complete. 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.879+0000 7f89189fa700 1 Processor -- start 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.879+0000 7f89189fa700 1 -- start start 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f891405e140 0x7f89141349f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f891412acc0 0x7f89141353d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914088890 con 0x7f891412acc0 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89141358c0 con 0x7f8914090b70 2024-04-25T11:01:00.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f891408aaa0 con 0x7f891405e140 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8912d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f891412acc0 0x7f89141353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914134ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:34936/0 (socket says 172.21.15.155:34936) 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 -- 172.21.15.155:0/3830508932 learned_addr learned my addr 172.21.15.155:0/3830508932 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f891259c700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f891405e140 0x7f89141349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 -- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f891405e140 msgr2=0x7f89141349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f891405e140 0x7f89141349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 -- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f891412acc0 msgr2=0x7f89141353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f891412acc0 0x7f89141353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 -- 172.21.15.155:0/3830508932 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c009ae0 con 0x7f8914090b70 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8912d9d700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f891412acc0 0x7f89141353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8911d9b700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914134ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8914065fc0 tx=0x7f8904017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:00.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400f890 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904006910 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400fea0 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.880+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89040092a0 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.881+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891408a4c0 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.881+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f890401b620 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.882+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8914047540 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.882+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f8904056a00 con 0x7f8914090b70 2024-04-25T11:01:00.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:00.885+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f8904024d70 con 0x7f8914090b70 2024-04-25T11:01:01.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.025+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f891412bf90 con 0x7f8914090b70 2024-04-25T11:01:01.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.025+0000 7f8902ffd700 1 -- 172.21.15.155:0/3830508932 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+8139 (secure 0 0 0) 0x7f890403abb0 con 0x7f8914090b70 2024-04-25T11:01:01.026 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:01.026 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":12,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:01:00.412304+0000","last_up_change":"2024-04-25T11:00:58.405672+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":43900}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":43900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":43900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":43900}]},"public_addr":"172.21.15.155:6800/43900","cluster_addr":"172.21.15.155:6801/43900","heartbeat_back_addr":"172.21.15.155:6803/43900","heartbeat_front_addr":"172.21.15.155:6802/43900","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 >> v1:172.21.15.155:6816/43298 conn(0x7f88f4041b80 legacy=0x7f88f4044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 msgr2=0x7f8914134ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914134ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8914065fc0 tx=0x7f8904017000).stop 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 shutdown_connections 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f891412acc0 0x7f89141353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f891405e140 0x7f89141349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 --2- 172.21.15.155:0/3830508932 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8914090b70 0x7f8914134ee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 >> 172.21.15.155:0/3830508932 conn(0x7f8914095700 msgr2=0x7f891412a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 shutdown_connections 2024-04-25T11:01:01.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.027+0000 7f89189fa700 1 -- 172.21.15.155:0/3830508932 wait complete. 2024-04-25T11:01:01.039 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:01:01.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:01:01.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 Processor -- start 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 -- start start 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 0x7f946c066270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 0x7f946c129490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c067be0 con 0x7f946c090460 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c067920 con 0x7f946c1270a0 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f947258f700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0912a0 con 0x7f946c065e70 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 0x7f946c066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.154+0000 7f9470b2c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 0x7f946c129490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f9470b2c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 0x7f946c129490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39844/0 (socket says 172.21.15.155:39844) 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 0x7f946c066270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50180/0 (socket says 172.21.15.155:50180) 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946bfff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f946c090460 0x7f946c065760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:60838/0 (socket says 172.21.15.155:60838) 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f9470b2c700 1 -- 172.21.15.155:0/1055885973 learned_addr learned my addr 172.21.15.155:0/1055885973 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946b7fe700 1 -- 172.21.15.155:0/1055885973 >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 msgr2=0x7f946c129490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946b7fe700 1 --2- 172.21.15.155:0/1055885973 >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 0x7f946c129490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946b7fe700 1 -- 172.21.15.155:0/1055885973 >> v1:172.21.15.155:6789/0 conn(0x7f946c090460 legacy=0x7f946c065760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946b7fe700 1 -- 172.21.15.155:0/1055885973 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c067550 con 0x7f946c065e70 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946affd700 1 -- 172.21.15.155:0/1055885973 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818083358 0 0) 0x7f946c067be0 con 0x7f946c090460 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f9470b2c700 1 --2- 172.21.15.155:0/1055885973 >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 0x7f946c129490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946b7fe700 1 --2- 172.21.15.155:0/1055885973 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 0x7f946c066270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f945c009760 tx=0x7f945c009060).ready entity=mon.2 client_cookie=72ef365221f367d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946affd700 1 -- 172.21.15.155:0/1055885973 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00f510 con 0x7f946c065e70 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946affd700 1 -- 172.21.15.155:0/1055885973 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00f670 con 0x7f946c065e70 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.155+0000 7f946affd700 1 -- 172.21.15.155:0/1055885973 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00f9f0 con 0x7f946c065e70 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 -- 172.21.15.155:0/1055885973 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 msgr2=0x7f946c066270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 --2- 172.21.15.155:0/1055885973 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 0x7f946c066270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f945c009760 tx=0x7f945c009060).stop 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 -- 172.21.15.155:0/1055885973 shutdown_connections 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 --2- 172.21.15.155:0/1055885973 >> v2:172.21.15.155:3300/0 conn(0x7f946c1270a0 0x7f946c129490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.158 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 --2- 172.21.15.155:0/1055885973 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c065e70 0x7f946c066270 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 -- 172.21.15.155:0/1055885973 >> 172.21.15.155:0/1055885973 conn(0x7f946c095700 msgr2=0x7f946c0915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 -- 172.21.15.155:0/1055885973 shutdown_connections 2024-04-25T11:01:01.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 -- 172.21.15.155:0/1055885973 wait complete. 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.156+0000 7f947258f700 1 Processor -- start 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 -- start start 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 0x7f946c11dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 0x7f946c120860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f946c1270a0 0x7f946c120d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c120160 con 0x7f946c065e70 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1205a0 con 0x7f946c1270a0 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f947258f700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f946c11f9f0 con 0x7f946c090460 2024-04-25T11:01:01.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 0x7f946c120860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 0x7f946c11dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 0x7f946c120860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:50192/0 (socket says 172.21.15.155:50192) 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946b7fe700 1 -- 172.21.15.155:0/3601586244 learned_addr learned my addr 172.21.15.155:0/3601586244 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 0x7f946c11dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39860/0 (socket says 172.21.15.155:39860) 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f9470b2c700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f946c1270a0 0x7f946c120d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 -- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 msgr2=0x7f946c120860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 0x7f946c120860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 -- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f946c1270a0 msgr2=0x7f946c120d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f946c1270a0 0x7f946c120d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946bfff700 1 -- 172.21.15.155:0/3601586244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464009ae0 con 0x7f946c065e70 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.157+0000 7f946b7fe700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 0x7f946c120860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.158+0000 7f946bfff700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 0x7f946c11dcc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9460000c00 tx=0x7f9460007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.158+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600070d0 con 0x7f946c065e70 2024-04-25T11:01:01.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.158+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460007250 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.158+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c0092a0 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.158+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460005670 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.158+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c121240 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.159+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f9460005a60 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.159+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f94600548c0 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.159+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c11fcd0 con 0x7f946c065e70 2024-04-25T11:01:01.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.163+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f9460018020 con 0x7f946c065e70 2024-04-25T11:01:01.303 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.303+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f946c047540 con 0x7f946c065e70 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.303+0000 7f9468ff9700 1 -- 172.21.15.155:0/3601586244 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+8139 (secure 0 0 0) 0x7f9460022070 con 0x7f946c065e70 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:01.304 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":12,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:01:00.412304+0000","last_up_change":"2024-04-25T11:00:58.405672+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":43900}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":43900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":43900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":43900}]},"public_addr":"172.21.15.155:6800/43900","cluster_addr":"172.21.15.155:6801/43900","heartbeat_back_addr":"172.21.15.155:6803/43900","heartbeat_front_addr":"172.21.15.155:6802/43900","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 >> v1:172.21.15.155:6816/43298 conn(0x7f9458041b80 legacy=0x7f9458044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 msgr2=0x7f946c11dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 0x7f946c11dcc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9460000c00 tx=0x7f9460007ac0).stop 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 shutdown_connections 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f946c1270a0 0x7f946c120d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f946c065e70 0x7f946c11dcc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 --2- 172.21.15.155:0/3601586244 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f946c090460 0x7f946c120860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 >> 172.21.15.155:0/3601586244 conn(0x7f946c095700 msgr2=0x7f946c068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 shutdown_connections 2024-04-25T11:01:01.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.305+0000 7f947258f700 1 -- 172.21.15.155:0/3601586244 wait complete. 2024-04-25T11:01:01.318 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:01:01.318 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:01:01.319 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:01:01.319 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:01:01.448 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.447+0000 7fdc10dbd700 1 Processor -- start 2024-04-25T11:01:01.448 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc10dbd700 1 -- start start 2024-04-25T11:01:01.449 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7f1421652700 1 Processor -- start 2024-04-25T11:01:01.449 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7f1421652700 1 -- start start 2024-04-25T11:01:01.449 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc10dbd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 0x7fdc0c129d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc10dbd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c066660 0x7fdc0c131ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc10dbd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0c1297d0 con 0x7fdc0c12a230 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc10dbd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c068770 con 0x7fdc0c129960 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc10dbd700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c067d80 con 0x7fdc0c066660 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc0a59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 0x7fdc0c129d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc0a59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 0x7fdc0c129d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49142/0 (socket says 172.21.15.155:49142) 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc0a59c700 1 -- 172.21.15.155:0/3034831427 learned_addr learned my addr 172.21.15.155:0/3034831427 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7fdc0ad9d700 1 --2- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c066660 0x7fdc0c131ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7f1421652700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 0x7f141c128970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.448+0000 7f1421652700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c128e60 0x7f141c12b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f1421652700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c123200 con 0x7f141c1235c0 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f1421652700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0698c0 con 0x7f141c126580 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f1421652700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1240b0 con 0x7f141c128e60 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 0x7f141c128970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 0x7f141c128970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49154/0 (socket says 172.21.15.155:49154) 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141a7fc700 1 -- 172.21.15.155:0/2922022163 learned_addr learned my addr 172.21.15.155:0/2922022163 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141b7fe700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c128e60 0x7f141c12b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7fdc0a59c700 1 -- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c066660 msgr2=0x7fdc0c131ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7fdc0a59c700 1 --2- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c066660 0x7fdc0c131ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7fdc0a59c700 1 -- 172.21.15.155:0/3034831427 >> v1:172.21.15.155:6789/0 conn(0x7fdc0c12a230 legacy=0x7fdc0c065f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7fdc0a59c700 1 -- 172.21.15.155:0/3034831427 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0676e0 con 0x7fdc0c129960 2024-04-25T11:01:01.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7fdc0a59c700 1 --2- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 0x7fdc0c129d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fdbfc009830 tx=0x7fdbfc0090f0).ready entity=mon.1 client_cookie=f0525a559c0ba5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141b7fe700 1 -- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 msgr2=0x7f141c128970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141b7fe700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 0x7f141c128970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141b7fe700 1 -- 172.21.15.155:0/2922022163 >> v1:172.21.15.155:6789/0 conn(0x7f141c1235c0 legacy=0x7f141c1239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141b7fe700 1 -- 172.21.15.155:0/2922022163 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c069d70 con 0x7f141c128e60 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141a7fc700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 0x7f141c128970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7f141b7fe700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c128e60 0x7f141c12b250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1414009e90 tx=0x7f1414009780).ready entity=mon.2 client_cookie=48bf2ee35c4307d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.449+0000 7fdc0959a700 1 -- 172.21.15.155:0/3034831427 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00b580 con 0x7fdc0c129960 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc0959a700 1 -- 172.21.15.155:0/3034831427 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc00b6e0 con 0x7fdc0c129960 2024-04-25T11:01:01.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc0959a700 1 -- 172.21.15.155:0/3034831427 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00ba80 con 0x7fdc0c129960 2024-04-25T11:01:01.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1419ffb700 1 -- 172.21.15.155:0/2922022163 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141400e070 con 0x7f141c128e60 2024-04-25T11:01:01.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1419ffb700 1 -- 172.21.15.155:0/2922022163 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141400bb20 con 0x7f141c128e60 2024-04-25T11:01:01.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1419ffb700 1 -- 172.21.15.155:0/2922022163 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141400bea0 con 0x7f141c128e60 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 -- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 msgr2=0x7fdc0c129d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 --2- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 0x7fdc0c129d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fdbfc009830 tx=0x7fdbfc0090f0).stop 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 -- 172.21.15.155:0/3034831427 shutdown_connections 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 -- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c128e60 msgr2=0x7f141c12b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 --2- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c066660 0x7fdc0c131ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 --2- 172.21.15.155:0/3034831427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c129960 0x7fdc0c129d40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 -- 172.21.15.155:0/3034831427 >> 172.21.15.155:0/3034831427 conn(0x7fdc0c0956f0 msgr2=0x7fdc0c095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c128e60 0x7f141c12b250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1414009e90 tx=0x7f1414009780).stop 2024-04-25T11:01:01.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 -- 172.21.15.155:0/2922022163 shutdown_connections 2024-04-25T11:01:01.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c128e60 0x7f141c12b250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 --2- 172.21.15.155:0/2922022163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c126580 0x7f141c128970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 -- 172.21.15.155:0/2922022163 >> 172.21.15.155:0/2922022163 conn(0x7f141c096b50 msgr2=0x7f141c091a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 -- 172.21.15.155:0/3034831427 shutdown_connections 2024-04-25T11:01:01.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7f1421652700 1 -- 172.21.15.155:0/2922022163 shutdown_connections 2024-04-25T11:01:01.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.450+0000 7fdc10dbd700 1 -- 172.21.15.155:0/3034831427 wait complete. 2024-04-25T11:01:01.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.451+0000 7f1421652700 1 -- 172.21.15.155:0/2922022163 wait complete. 2024-04-25T11:01:01.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.451+0000 7f1421652700 1 Processor -- start 2024-04-25T11:01:01.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 -- start start 2024-04-25T11:01:01.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7fdc10dbd700 1 Processor -- start 2024-04-25T11:01:01.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7fdc10dbd700 1 -- start start 2024-04-25T11:01:01.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c1235c0 0x7f141c10ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c126580 0x7f141c10f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c128e60 0x7f141c10f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c10fce0 con 0x7f141c128e60 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c110b20 con 0x7f141c1235c0 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f1421652700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f141c10fec0 con 0x7f141c126580 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f141b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c128e60 0x7f141c10f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f141b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c128e60 0x7f141c10f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34138/0 (socket says 172.21.15.155:34138) 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.452+0000 7f141b7fe700 1 -- 172.21.15.155:0/4106464643 learned_addr learned my addr 172.21.15.155:0/4106464643 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7f141b7fe700 1 -- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c126580 msgr2=0x7f141c10f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7f141b7fe700 1 --2- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f141c126580 0x7f141c10f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7f141b7fe700 1 -- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c1235c0 msgr2=0x7f141c10ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7f141b7fe700 1 --2- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f141c1235c0 0x7f141c10ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7f141b7fe700 1 -- 172.21.15.155:0/4106464643 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410009370 con 0x7f141c128e60 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc10dbd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc0c066660 0x7fdc0c139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc10dbd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c129960 0x7fdc0c139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc10dbd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c12a230 0x7fdc0c11a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc10dbd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c11df70 con 0x7fdc0c066660 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc10dbd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c11d8e0 con 0x7fdc0c12a230 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc10dbd700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c11dcb0 con 0x7fdc0c129960 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c129960 0x7fdc0c139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c129960 0x7fdc0c139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47130/0 (socket says 172.21.15.155:47130) 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc0ad9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c12a230 0x7fdc0c11a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 -- 172.21.15.155:0/1666704130 learned_addr learned my addr 172.21.15.155:0/1666704130 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc0a59c700 1 --2- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc0c066660 0x7fdc0c139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 -- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c12a230 msgr2=0x7fdc0c11a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 --2- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc0c12a230 0x7fdc0c11a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 -- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc0c066660 msgr2=0x7fdc0c139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 --2- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc0c066660 0x7fdc0c139180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 -- 172.21.15.155:0/1666704130 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc009370 con 0x7fdc0c129960 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdc09d9b700 1 --2- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c129960 0x7fdc0c139670 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fdc0000c4f0 tx=0x7fdc00007dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000ce00 con 0x7fdc0c129960 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00007020 con 0x7fdc0c129960 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000073c0 con 0x7fdc0c129960 2024-04-25T11:01:01.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.453+0000 7f141b7fe700 1 --2- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c128e60 0x7f141c10f7f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f1414011000 tx=0x7f1414009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c11c030 con 0x7fdc0c129960 2024-04-25T11:01:01.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c139b60 con 0x7fdc0c129960 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141401d630 con 0x7f141c128e60 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141401d790 con 0x7f141c128e60 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141401db50 con 0x7f141c128e60 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1414009a20 con 0x7f141c128e60 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.454+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c111bd0 con 0x7f141c128e60 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.455+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc0c139e40 con 0x7fdc0c129960 2024-04-25T11:01:01.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.455+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f141c047540 con 0x7f141c128e60 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.456+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f1414003040 con 0x7f141c128e60 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.455+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7fdc000068b0 con 0x7fdc0c129960 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.455+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fdc00054510 con 0x7fdc0c129960 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.455+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 --> v1:172.21.15.155:6808/43899 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbec04bfa0 con 0x7fdbec0482b0 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.457+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== osd.1 v1:172.21.15.155:6808/43899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fdbec04bfa0 con 0x7fdbec0482b0 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.457+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f1414006720 con 0x7f141c128e60 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.457+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 --> v1:172.21.15.155:6812/43902 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f140804bfa0 con 0x7f14080482b0 2024-04-25T11:01:01.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.457+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1414006350 con 0x7f141c128e60 2024-04-25T11:01:01.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.458+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc00035020 con 0x7fdc0c129960 2024-04-25T11:01:01.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.459+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== osd.2 v1:172.21.15.155:6812/43902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f140804bfa0 con 0x7f14080482b0 2024-04-25T11:01:01.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 Processor -- start 2024-04-25T11:01:01.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 -- start start 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b0090460 0x7ff0b0065760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b0066050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff0b008e050 0x7ff0b008e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b0067be0 con 0x7ff0b008e050 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0067920 con 0x7ff0b0090460 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.460+0000 7ff0b6493700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0066540 con 0x7ff0b0065c50 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff0b008e050 0x7ff0b008e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff0b008e050 0x7ff0b008e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34152/0 (socket says 172.21.15.155:34152) 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 -- 172.21.15.155:0/1501579955 learned_addr learned my addr 172.21.15.155:0/1501579955 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0af7fe700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b0066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0affff700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b0090460 0x7ff0b0065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 -- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 msgr2=0x7ff0b0066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b0066050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 -- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b0090460 msgr2=0x7ff0b0065760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b0090460 0x7ff0b0065760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 -- 172.21.15.155:0/1501579955 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b00912a0 con 0x7ff0b008e050 2024-04-25T11:01:01.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0af7fe700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b0066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0b4a30700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b008e050 0x7ff0b008e500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff0a800bfc0 tx=0x7ff0a800d040).ready entity=mon.0 client_cookie=a1f7833a79b591bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7fcbc666d700 1 Processor -- start 2024-04-25T11:01:01.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0aeffd700 1 -- 172.21.15.155:0/1501579955 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800a2e0 con 0x7ff0b008e050 2024-04-25T11:01:01.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7fcbc666d700 1 -- start start 2024-04-25T11:01:01.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0aeffd700 1 -- 172.21.15.155:0/1501579955 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a800b030 con 0x7ff0b008e050 2024-04-25T11:01:01.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.461+0000 7ff0aeffd700 1 -- 172.21.15.155:0/1501579955 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800b3d0 con 0x7ff0b008e050 2024-04-25T11:01:01.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 -- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b008e050 msgr2=0x7ff0b008e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b008e050 0x7ff0b008e500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff0a800bfc0 tx=0x7ff0a800d040).stop 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 -- 172.21.15.155:0/1501579955 shutdown_connections 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b008e050 0x7ff0b008e500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b0090460 0x7ff0b0065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 --2- 172.21.15.155:0/1501579955 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b0066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7ff0b6493700 1 -- 172.21.15.155:0/1501579955 >> 172.21.15.155:0/1501579955 conn(0x7ff0b0095700 msgr2=0x7ff0b00915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbc666d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcbc0090460 0x7fcbc005e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbc666d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc0123080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbc666d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc0067be0 con 0x7fcbc0065760 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbc666d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc0067920 con 0x7fcbc0090460 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbc666d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00912a0 con 0x7fcbc0066270 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbbffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcbc0090460 0x7fcbc005e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbbffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcbc0090460 0x7fcbc005e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34158/0 (socket says 172.21.15.155:34158) 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.462+0000 7fcbbffff700 1 -- 172.21.15.155:0/777761052 learned_addr learned my addr 172.21.15.155:0/777761052 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7fcbbffff700 1 -- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 msgr2=0x7fcbc0123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7fcbbffff700 1 --2- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc0123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7fcbbffff700 1 -- 172.21.15.155:0/777761052 >> v1:172.21.15.155:6789/0 conn(0x7fcbc0065760 legacy=0x7fcbc0065b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7fcbbffff700 1 -- 172.21.15.155:0/777761052 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0067550 con 0x7fcbc0090460 2024-04-25T11:01:01.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7fcbbffff700 1 --2- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0090460 0x7fcbc005e140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcbb800ffc0 tx=0x7fcbb800fb90).ready entity=mon.0 client_cookie=3989f608bac877ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7ff0b6493700 1 -- 172.21.15.155:0/1501579955 shutdown_connections 2024-04-25T11:01:01.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.463+0000 7ff0b6493700 1 -- 172.21.15.155:0/1501579955 wait complete. 2024-04-25T11:01:01.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.464+0000 7ff0b6493700 1 Processor -- start 2024-04-25T11:01:01.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7fcbbeffd700 1 -- 172.21.15.155:0/777761052 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800d3d0 con 0x7fcbc0090460 2024-04-25T11:01:01.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 -- start start 2024-04-25T11:01:01.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7fcbbeffd700 1 -- 172.21.15.155:0/777761052 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb800d530 con 0x7fcbc0090460 2024-04-25T11:01:01.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7fcbbeffd700 1 -- 172.21.15.155:0/777761052 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800d6a0 con 0x7fcbc0090460 2024-04-25T11:01:01.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b008a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b008e050 0x7ff0b008a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b0090460 0x7ff0b008aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b008b390 con 0x7ff0b0090460 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b008c1b0 con 0x7ff0b008e050 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b6493700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b008b570 con 0x7ff0b0065c50 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0affff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b008a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0affff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b008a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47158/0 (socket says 172.21.15.155:47158) 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0affff700 1 -- 172.21.15.155:0/1113255024 learned_addr learned my addr 172.21.15.155:0/1113255024 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0af7fe700 1 --2- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b008e050 0x7ff0b008a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7ff0b4a30700 1 --2- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b0090460 0x7ff0b008aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7fcbc666d700 1 -- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0090460 msgr2=0x7fcbc005e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.465+0000 7fcbc666d700 1 --2- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0090460 0x7fcbc005e140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcbb800ffc0 tx=0x7fcbb800fb90).stop 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7fcbc666d700 1 -- 172.21.15.155:0/777761052 shutdown_connections 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7fcbc666d700 1 --2- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc0123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7fcbc666d700 1 --2- 172.21.15.155:0/777761052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0090460 0x7fcbc005e140 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0x7fcbb800ffc0 tx=0x7fcbb800fb90).stop 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7fcbc666d700 1 -- 172.21.15.155:0/777761052 >> 172.21.15.155:0/777761052 conn(0x7fcbc0095700 msgr2=0x7fcbc00915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0af7fe700 1 -- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 msgr2=0x7ff0b008a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0af7fe700 1 --2- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff0b0065c50 0x7ff0b008a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0af7fe700 1 -- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b0090460 msgr2=0x7ff0b008aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0af7fe700 1 --2- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0b0090460 0x7ff0b008aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0af7fe700 1 -- 172.21.15.155:0/1113255024 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a400d740 con 0x7ff0b008e050 2024-04-25T11:01:01.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7fcbc666d700 1 -- 172.21.15.155:0/777761052 shutdown_connections 2024-04-25T11:01:01.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0af7fe700 1 --2- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b008e050 0x7ff0b008a9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff0a4012c70 tx=0x7ff0a400f040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400d2e0 con 0x7ff0b008e050 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a4013040 con 0x7ff0b008e050 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4010830 con 0x7ff0b008e050 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a800bb00 con 0x7ff0b008e050 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7fcbc666d700 1 -- 172.21.15.155:0/777761052 wait complete. 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.467+0000 7fcbc666d700 1 Processor -- start 2024-04-25T11:01:01.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.467+0000 7fcbc666d700 1 -- start start 2024-04-25T11:01:01.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.466+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b008d440 con 0x7ff0b008e050 2024-04-25T11:01:01.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.467+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7ff0a4010ba0 con 0x7ff0b008e050 2024-04-25T11:01:01.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc666d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0065760 0x7fcbc008bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc666d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc008c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.480 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7ff0a4058e60 con 0x7ff0b008e050 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc008c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc008c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47172/0 (socket says 172.21.15.155:47172) 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc666d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbc008e9b0 0x7fcbc008c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc666d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc008d940 con 0x7fcbc0065760 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc666d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc008cd20 con 0x7fcbc008e9b0 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc666d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc0119cb0 con 0x7fcbc0066270 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 -- 172.21.15.155:0/2471475312 learned_addr learned my addr 172.21.15.155:0/2471475312 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbc4c0a700 1 --2- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbc008e9b0 0x7fcbc008c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 -- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbc008e9b0 msgr2=0x7fcbc008c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 --2- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbc008e9b0 0x7fcbc008c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 -- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0065760 msgr2=0x7fcbc008bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 --2- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc0065760 0x7fcbc008bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7fcbbf7fe700 1 -- 172.21.15.155:0/2471475312 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb800fe30 con 0x7fcbc0066270 2024-04-25T11:01:01.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.468+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 --> v1:172.21.15.155:6804/43901 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0b005a130 con 0x7ff0b006c530 2024-04-25T11:01:01.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.469+0000 7fcbbf7fe700 1 --2- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc008c160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fcbb400c4c0 tx=0x7fcbb4007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.469+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4007020 con 0x7fcbc0066270 2024-04-25T11:01:01.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.469+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4001e90 con 0x7fcbc0066270 2024-04-25T11:01:01.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.469+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40073d0 con 0x7fcbc0066270 2024-04-25T11:01:01.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.469+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc0130ad0 con 0x7fcbc0066270 2024-04-25T11:01:01.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.469+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc0130fc0 con 0x7fcbc0066270 2024-04-25T11:01:01.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.470+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== osd.3 v1:172.21.15.155:6804/43901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7ff0a801eac0 con 0x7ff0b006c530 2024-04-25T11:01:01.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.470+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcbc0047540 con 0x7fcbc0066270 2024-04-25T11:01:01.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.470+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7fcbb40068b0 con 0x7fcbc0066270 2024-04-25T11:01:01.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.470+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fcbb4054a40 con 0x7fcbc0066270 2024-04-25T11:01:01.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.470+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 --> v1:172.21.15.155:6800/43900 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbac04bfa0 con 0x7fcbac0482b0 2024-04-25T11:01:01.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.470+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbb4059220 con 0x7fcbc0066270 2024-04-25T11:01:01.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.471+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== osd.0 v1:172.21.15.155:6800/43900 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fcbac04bfa0 con 0x7fcbac0482b0 2024-04-25T11:01:01.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.484+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 --> v1:172.21.15.155:6808/43899 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc0c047540 con 0x7fdbec0482b0 2024-04-25T11:01:01.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.484+0000 7fdbfaffd700 1 -- 172.21.15.155:0/1666704130 <== osd.1 v1:172.21.15.155:6808/43899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 691802991) 0x7fdc0c047540 con 0x7fdbec0482b0 2024-04-25T11:01:01.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.484+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 >> v1:172.21.15.155:6808/43899 conn(0x7fdbec0482b0 legacy=0x7fdbec04a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.484+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 --> v1:172.21.15.155:6804/43901 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0b0131290 con 0x7ff0b006c530 2024-04-25T11:01:01.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 >> v1:172.21.15.155:6816/43298 conn(0x7fdbec041b80 legacy=0x7fdbec044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c129960 msgr2=0x7fdc0c139670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 --2- 172.21.15.155:0/1666704130 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdc0c129960 0x7fdc0c139670 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fdc0000c4f0 tx=0x7fdc00007dd0).stop 2024-04-25T11:01:01.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc0ad9d700 1 -- 172.21.15.155:0/1666704130 reap_dead start 2024-04-25T11:01:01.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0acff9700 1 -- 172.21.15.155:0/1113255024 <== osd.3 v1:172.21.15.155:6804/43901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2217796425) 0x7ff0b0131290 con 0x7ff0b006c530 2024-04-25T11:01:01.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 shutdown_connections 2024-04-25T11:01:01.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 >> 172.21.15.155:0/1666704130 conn(0x7fdc0c0956f0 msgr2=0x7fdc0c0932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 shutdown_connections 2024-04-25T11:01:01.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7fdc10dbd700 1 -- 172.21.15.155:0/1666704130 wait complete. 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 >> v1:172.21.15.155:6804/43901 conn(0x7ff0b006c530 legacy=0x7ff0b01308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 >> v1:172.21.15.155:6816/43298 conn(0x7ff09c041bd0 legacy=0x7ff09c044090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b008e050 msgr2=0x7ff0b008a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 --2- 172.21.15.155:0/1113255024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0b008e050 0x7ff0b008a9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff0a4012c70 tx=0x7ff0a400f040).stop 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b4a30700 1 -- 172.21.15.155:0/1113255024 reap_dead start 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 shutdown_connections 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 >> 172.21.15.155:0/1113255024 conn(0x7ff0b0095700 msgr2=0x7ff0b0093570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 shutdown_connections 2024-04-25T11:01:01.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.485+0000 7ff0b6493700 1 -- 172.21.15.155:0/1113255024 wait complete. 2024-04-25T11:01:01.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.486+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 --> v1:172.21.15.155:6800/43900 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcbc006c790 con 0x7fcbac0482b0 2024-04-25T11:01:01.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.486+0000 7fcbbcff9700 1 -- 172.21.15.155:0/2471475312 <== osd.0 v1:172.21.15.155:6800/43900 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2217796425) 0x7fcbc006c790 con 0x7fcbac0482b0 2024-04-25T11:01:01.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 >> v1:172.21.15.155:6800/43900 conn(0x7fcbac0482b0 legacy=0x7fcbac04a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 >> v1:172.21.15.155:6816/43298 conn(0x7fcbac041b80 legacy=0x7fcbac044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 msgr2=0x7fcbc008c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 --2- 172.21.15.155:0/2471475312 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fcbc0066270 0x7fcbc008c160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fcbb400c4c0 tx=0x7fcbb4007e00).stop 2024-04-25T11:01:01.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc4c0a700 1 -- 172.21.15.155:0/2471475312 reap_dead start 2024-04-25T11:01:01.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 --> v1:172.21.15.155:6812/43902 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f141c06c790 con 0x7f14080482b0 2024-04-25T11:01:01.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 shutdown_connections 2024-04-25T11:01:01.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 >> 172.21.15.155:0/2471475312 conn(0x7fcbc0095700 msgr2=0x7fcbc0084fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 shutdown_connections 2024-04-25T11:01:01.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7fcbc666d700 1 -- 172.21.15.155:0/2471475312 wait complete. 2024-04-25T11:01:01.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.487+0000 7f14037fe700 1 -- 172.21.15.155:0/4106464643 <== osd.2 v1:172.21.15.155:6812/43902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3679948908) 0x7f141c06c790 con 0x7f14080482b0 2024-04-25T11:01:01.492 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.489+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 >> v1:172.21.15.155:6812/43902 conn(0x7f14080482b0 legacy=0x7f140804a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.492 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.489+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 >> v1:172.21.15.155:6816/43298 conn(0x7f1408041b80 legacy=0x7f1408044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.492 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.489+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c128e60 msgr2=0x7f141c10f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.492 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.489+0000 7f1421652700 1 --2- 172.21.15.155:0/4106464643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141c128e60 0x7f141c10f7f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f1414011000 tx=0x7f1414009780).stop 2024-04-25T11:01:01.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.489+0000 7f141b7fe700 1 -- 172.21.15.155:0/4106464643 reap_dead start 2024-04-25T11:01:01.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.492+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 shutdown_connections 2024-04-25T11:01:01.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.492+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 >> 172.21.15.155:0/4106464643 conn(0x7f141c096b50 msgr2=0x7f141c095bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.492+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 shutdown_connections 2024-04-25T11:01:01.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.492+0000 7f1421652700 1 -- 172.21.15.155:0/4106464643 wait complete. 2024-04-25T11:01:01.502 INFO:teuthology.orchestra.run.smithi155.stdout:38654705667 2024-04-25T11:01:01.503 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:01:01.504 INFO:teuthology.orchestra.run.smithi155.stdout:42949672962 2024-04-25T11:01:01.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:01:01.505 INFO:teuthology.orchestra.run.smithi155.stdout:38654705667 2024-04-25T11:01:01.505 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:01:01.507 INFO:teuthology.orchestra.run.smithi155.stdout:42949672963 2024-04-25T11:01:01.507 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:01:01.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 Processor -- start 2024-04-25T11:01:01.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 -- start start 2024-04-25T11:01:01.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c124b50 0x7f139c126f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f139c127430 0x7f139c129820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c069490 con 0x7f139c067b10 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139c068840 con 0x7f139c127430 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.635+0000 7f13a47a9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0696d0 con 0x7f139c124b50 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2545700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f139c067b10 0x7f139c11e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44650/0 (socket says 172.21.15.155:44650) 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2d46700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f139c127430 0x7f139c129820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2545700 1 -- 172.21.15.155:0/3392172856 learned_addr learned my addr 172.21.15.155:0/3392172856 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a1d44700 1 --2- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c124b50 0x7f139c126f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a1543700 1 -- 172.21.15.155:0/3392172856 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354716487 0 0) 0x7f139c069490 con 0x7f139c067b10 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a1543700 1 -- 172.21.15.155:0/3392172856 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1388005600 con 0x7f139c067b10 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2d46700 1 -- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c124b50 msgr2=0x7f139c126f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2d46700 1 --2- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c124b50 0x7f139c126f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2d46700 1 -- 172.21.15.155:0/3392172856 >> v1:172.21.15.155:6789/0 conn(0x7f139c067b10 legacy=0x7f139c11e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2d46700 1 -- 172.21.15.155:0/3392172856 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c068fb0 con 0x7f139c127430 2024-04-25T11:01:01.639 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a2d46700 1 --2- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c127430 0x7f139c129820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1394009e60 tx=0x7f1394009780).ready entity=mon.0 client_cookie=246457b05de11180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.639 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.636+0000 7f13a1543700 1 -- 172.21.15.155:0/3392172856 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139400e070 con 0x7f139c127430 2024-04-25T11:01:01.639 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a1543700 1 -- 172.21.15.155:0/3392172856 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139400bb60 con 0x7f139c127430 2024-04-25T11:01:01.639 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a1543700 1 -- 172.21.15.155:0/3392172856 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394012410 con 0x7f139c127430 2024-04-25T11:01:01.641 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 -- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c127430 msgr2=0x7f139c129820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.641 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c127430 0x7f139c129820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1394009e60 tx=0x7f1394009780).stop 2024-04-25T11:01:01.641 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 -- 172.21.15.155:0/3392172856 shutdown_connections 2024-04-25T11:01:01.641 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c127430 0x7f139c129820 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.641 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3392172856 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c124b50 0x7f139c126f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.641 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 -- 172.21.15.155:0/3392172856 >> 172.21.15.155:0/3392172856 conn(0x7f139c094420 msgr2=0x7f139c094830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 -- 172.21.15.155:0/3392172856 shutdown_connections 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.637+0000 7f13a47a9700 1 -- 172.21.15.155:0/3392172856 wait complete. 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.638+0000 7f13a47a9700 1 Processor -- start 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.638+0000 7f13a47a9700 1 -- start start 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a47a9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f139c067b10 0x7f139c134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a47a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 0x7f139c134f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a47a9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c127430 0x7f139c118f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a47a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c124800 con 0x7f139c124b50 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a47a9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139c10f3d0 con 0x7f139c067b10 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a47a9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f139c11a900 con 0x7f139c127430 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a1d44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 0x7f139c134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a1d44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 0x7f139c134f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34194/0 (socket says 172.21.15.155:34194) 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a1d44700 1 -- 172.21.15.155:0/3670996381 learned_addr learned my addr 172.21.15.155:0/3670996381 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c127430 0x7f139c118f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 -- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f139c067b10 msgr2=0x7f139c134aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f139c067b10 0x7f139c134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 -- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 msgr2=0x7f139c134f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 0x7f139c134f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 -- 172.21.15.155:0/3670996381 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394009a20 con 0x7f139c127430 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a1d44700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 0x7f139c134f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.639+0000 7f13a2d46700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c127430 0x7f139c118f00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1394014000 tx=0x7f1394009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.640+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139401d630 con 0x7f139c127430 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.640+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139401d790 con 0x7f139c127430 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.640+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139401db50 con 0x7f139c127430 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.640+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c10f110 con 0x7f139c127430 2024-04-25T11:01:01.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.640+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c135480 con 0x7f139c127430 2024-04-25T11:01:01.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.641+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f1394003040 con 0x7f139c127430 2024-04-25T11:01:01.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.641+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f1394056ec0 con 0x7f139c127430 2024-04-25T11:01:01.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.642+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139c047540 con 0x7f139c127430 2024-04-25T11:01:01.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.646+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f139400a7e0 con 0x7f139c127430 2024-04-25T11:01:01.674 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.673+0000 7f7360813700 1 Processor -- start 2024-04-25T11:01:01.674 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.673+0000 7f7360813700 1 -- start start 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f7360813700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122b20 0x7f735c1273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f7360813700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f735c1278b0 0x7f735c129ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f7360813700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c069b40 con 0x7f735c122030 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f7360813700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c069020 con 0x7f735c1278b0 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f7360813700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c068890 con 0x7f735c122b20 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f7359d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122b20 0x7f735c1273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f735a59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f735c122030 0x7f735c122410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44656/0 (socket says 172.21.15.155:44656) 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f735a59c700 1 -- 172.21.15.155:0/3916228567 learned_addr learned my addr 172.21.15.155:0/3916228567 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f735ad9d700 1 --2- 172.21.15.155:0/3916228567 >> v2:172.21.15.155:3300/0 conn(0x7f735c1278b0 0x7f735c129ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.674+0000 7f735959a700 1 -- 172.21.15.155:0/3916228567 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323897973 0 0) 0x7f735c069b40 con 0x7f735c122030 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f735959a700 1 -- 172.21.15.155:0/3916228567 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73440033f0 con 0x7f735c122030 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f7359d9b700 1 -- 172.21.15.155:0/3916228567 >> v2:172.21.15.155:3300/0 conn(0x7f735c1278b0 msgr2=0x7f735c129ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f7359d9b700 1 --2- 172.21.15.155:0/3916228567 >> v2:172.21.15.155:3300/0 conn(0x7f735c1278b0 0x7f735c129ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f7359d9b700 1 -- 172.21.15.155:0/3916228567 >> v1:172.21.15.155:6789/0 conn(0x7f735c122030 legacy=0x7f735c122410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f7359d9b700 1 -- 172.21.15.155:0/3916228567 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c069880 con 0x7f735c122b20 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f735ad9d700 1 --2- 172.21.15.155:0/3916228567 >> v2:172.21.15.155:3300/0 conn(0x7f735c1278b0 0x7f735c129ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f7359d9b700 1 --2- 172.21.15.155:0/3916228567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122b20 0x7f735c1273c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7350009760 tx=0x7f7350009060).ready entity=mon.1 client_cookie=1d96bf4b31774e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f735959a700 1 -- 172.21.15.155:0/3916228567 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500090d0 con 0x7f735c122b20 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f735959a700 1 -- 172.21.15.155:0/3916228567 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735000f510 con 0x7f735c122b20 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.675+0000 7f735959a700 1 -- 172.21.15.155:0/3916228567 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735000f890 con 0x7f735c122b20 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 -- 172.21.15.155:0/3916228567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122b20 msgr2=0x7f735c1273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 --2- 172.21.15.155:0/3916228567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122b20 0x7f735c1273c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7350009760 tx=0x7f7350009060).stop 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 -- 172.21.15.155:0/3916228567 shutdown_connections 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 --2- 172.21.15.155:0/3916228567 >> v2:172.21.15.155:3300/0 conn(0x7f735c1278b0 0x7f735c129ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 --2- 172.21.15.155:0/3916228567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122b20 0x7f735c1273c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 -- 172.21.15.155:0/3916228567 >> 172.21.15.155:0/3916228567 conn(0x7f735c0956f0 msgr2=0x7f735c095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 -- 172.21.15.155:0/3916228567 shutdown_connections 2024-04-25T11:01:01.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.676+0000 7f7360813700 1 -- 172.21.15.155:0/3916228567 wait complete. 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.677+0000 7f7360813700 1 Processor -- start 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.677+0000 7f7360813700 1 -- start start 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7360813700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 0x7f735c132ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7360813700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f735c122b20 0x7f735c132fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7360813700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f735c1278b0 0x7f735c10b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7360813700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c132750 con 0x7f735c1278b0 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7360813700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c11fa20 con 0x7f735c122030 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7360813700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f735c10e0e0 con 0x7f735c122b20 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 0x7f735c132ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 0x7f735c132ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49248/0 (socket says 172.21.15.155:49248) 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 -- 172.21.15.155:0/1864426017 learned_addr learned my addr 172.21.15.155:0/1864426017 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735ad9d700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f735c1278b0 0x7f735c10b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f7359d9b700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f735c122b20 0x7f735c132fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 -- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f735c122b20 msgr2=0x7f735c132fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f735c122b20 0x7f735c132fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 -- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f735c1278b0 msgr2=0x7f735c10b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f735c1278b0 0x7f735c10b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 -- 172.21.15.155:0/1864426017 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73500092a0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.678+0000 7f735a59c700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 0x7f735c132ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f734c00c510 tx=0x7f734c007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.679+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0072e0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.679+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c007440 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.679+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0057a0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.679+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c10e4b0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.679+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c1334b0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.680+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f734c005bd0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.680+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f734c0549b0 con 0x7f735c122030 2024-04-25T11:01:01.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.680+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c062aa0 con 0x7f735c122030 2024-04-25T11:01:01.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 Processor -- start 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 -- start start 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122390 0x7f70fc122770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f70fc090b70 0x7f70fc12a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc1221d0 con 0x7f70fc122c60 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc067ba0 con 0x7f70fc090b70 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f71014ad700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc05e300 con 0x7f70fc122390 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.681+0000 7f70fa7fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f70fc122c60 0x7f70fc090460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44660/0 (socket says 172.21.15.155:44660) 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70fa7fc700 1 -- 172.21.15.155:0/1972410637 learned_addr learned my addr 172.21.15.155:0/1972410637 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70faffd700 1 --2- 172.21.15.155:0/1972410637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122390 0x7f70fc122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70fb7fe700 1 --2- 172.21.15.155:0/1972410637 >> v2:172.21.15.155:3300/0 conn(0x7f70fc090b70 0x7f70fc12a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70f9ffb700 1 -- 172.21.15.155:0/1972410637 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197502591 0 0) 0x7f70fc1221d0 con 0x7f70fc122c60 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70f9ffb700 1 -- 172.21.15.155:0/1972410637 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e40033f0 con 0x7f70fc122c60 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70faffd700 1 -- 172.21.15.155:0/1972410637 >> v2:172.21.15.155:3300/0 conn(0x7f70fc090b70 msgr2=0x7f70fc12a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.684 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70faffd700 1 --2- 172.21.15.155:0/1972410637 >> v2:172.21.15.155:3300/0 conn(0x7f70fc090b70 0x7f70fc12a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70faffd700 1 -- 172.21.15.155:0/1972410637 >> v1:172.21.15.155:6789/0 conn(0x7f70fc122c60 legacy=0x7f70fc090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70faffd700 1 -- 172.21.15.155:0/1972410637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc067430 con 0x7f70fc122390 2024-04-25T11:01:01.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70fb7fe700 1 --2- 172.21.15.155:0/1972410637 >> v2:172.21.15.155:3300/0 conn(0x7f70fc090b70 0x7f70fc12a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.682+0000 7f70faffd700 1 --2- 172.21.15.155:0/1972410637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122390 0x7f70fc122770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f70ec009760 tx=0x7f70ec006f40).ready entity=mon.1 client_cookie=36f1a3fa08c30fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 Processor -- start 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 -- start start 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f009c0626c0 0x7f009c125460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 0x7f009c12cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c0676e0 con 0x7f009c067bc0 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c067870 con 0x7f009c0626c0 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a3b9d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f009c126060 con 0x7f009c12a6c0 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a213a700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 0x7f009c12cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a213a700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 0x7f009c12cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47218/0 (socket says 172.21.15.155:47218) 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a213a700 1 -- 172.21.15.155:0/3411472133 learned_addr learned my addr 172.21.15.155:0/3411472133 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.683+0000 7f00a1939700 1 --2- 172.21.15.155:0/3411472133 >> v2:172.21.15.155:3300/0 conn(0x7f009c0626c0 0x7f009c125460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a1939700 1 -- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 msgr2=0x7f009c12cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a1939700 1 --2- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 0x7f009c12cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a1939700 1 -- 172.21.15.155:0/3411472133 >> v1:172.21.15.155:6789/0 conn(0x7f009c067bc0 legacy=0x7f009c125950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a1939700 1 -- 172.21.15.155:0/3411472133 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c1261d0 con 0x7f009c0626c0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a0937700 1 -- 172.21.15.155:0/3411472133 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354519358 0 0) 0x7f009c0676e0 con 0x7f009c067bc0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a213a700 1 --2- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 0x7f009c12cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a1939700 1 --2- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c0626c0 0x7f009c125460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0098009760 tx=0x7f0098009060).ready entity=mon.0 client_cookie=ca6070a1d987584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a0937700 1 -- 172.21.15.155:0/3411472133 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800f510 con 0x7f009c0626c0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a0937700 1 -- 172.21.15.155:0/3411472133 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009800f670 con 0x7f009c0626c0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.684+0000 7f00a0937700 1 -- 172.21.15.155:0/3411472133 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800f9f0 con 0x7f009c0626c0 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c0626c0 msgr2=0x7f009c125460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c0626c0 0x7f009c125460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0098009760 tx=0x7f0098009060).stop 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3411472133 shutdown_connections 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c12a6c0 0x7f009c12cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3411472133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c0626c0 0x7f009c125460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3411472133 >> 172.21.15.155:0/3411472133 conn(0x7f009c0956f0 msgr2=0x7f009c095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3411472133 shutdown_connections 2024-04-25T11:01:01.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3411472133 wait complete. 2024-04-25T11:01:01.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f70f9ffb700 1 -- 172.21.15.155:0/1972410637 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec005880 con 0x7f70fc122390 2024-04-25T11:01:01.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f70f9ffb700 1 -- 172.21.15.155:0/1972410637 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec0059e0 con 0x7f70fc122390 2024-04-25T11:01:01.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f70f9ffb700 1 -- 172.21.15.155:0/1972410637 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec005d60 con 0x7f70fc122390 2024-04-25T11:01:01.690 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.685+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f734c035580 con 0x7f735c122030 2024-04-25T11:01:01.690 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 Processor -- start 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 -- start start 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 -- 172.21.15.155:0/1972410637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122390 msgr2=0x7f70fc122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 --2- 172.21.15.155:0/1972410637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122390 0x7f70fc122770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f70ec009760 tx=0x7f70ec006f40).stop 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 -- 172.21.15.155:0/1972410637 shutdown_connections 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 --2- 172.21.15.155:0/1972410637 >> v2:172.21.15.155:3300/0 conn(0x7f70fc090b70 0x7f70fc12a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 --2- 172.21.15.155:0/1972410637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122390 0x7f70fc122770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 -- 172.21.15.155:0/1972410637 >> 172.21.15.155:0/1972410637 conn(0x7f70fc095700 msgr2=0x7f70fc0915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 0x7f009c134a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c067bc0 0x7f009c134f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 0x7f009c1353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c121f50 con 0x7f009c067bc0 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c121320 con 0x7f009c12a6c0 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a3b9d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f009c126f40 con 0x7f009c0626c0 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a1939700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 0x7f009c134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a213a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 0x7f009c1353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a1939700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 0x7f009c134a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47232/0 (socket says 172.21.15.155:47232) 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a213a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 0x7f009c1353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49260/0 (socket says 172.21.15.155:49260) 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f00a1939700 1 -- 172.21.15.155:0/3610371660 learned_addr learned my addr 172.21.15.155:0/3610371660 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 -- 172.21.15.155:0/1972410637 shutdown_connections 2024-04-25T11:01:01.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.686+0000 7f71014ad700 1 -- 172.21.15.155:0/1972410637 wait complete. 2024-04-25T11:01:01.694 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a1138700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c067bc0 0x7f009c134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.694 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a213a700 1 -- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 msgr2=0x7f009c134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a213a700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 0x7f009c134a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a213a700 1 -- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c067bc0 msgr2=0x7f009c134f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a213a700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c067bc0 0x7f009c134f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a213a700 1 -- 172.21.15.155:0/3610371660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0094009ae0 con 0x7f009c12a6c0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a213a700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 0x7f009c1353f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0094009f10 tx=0x7f009400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940092e0 con 0x7f009c12a6c0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400f040 con 0x7f009c12a6c0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400cc20 con 0x7f009c12a6c0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00980092a0 con 0x7f009c12a6c0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.687+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c126b70 con 0x7f009c12a6c0 2024-04-25T11:01:01.695 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.688+0000 7f00a1939700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 0x7f009c134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:01.696 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.688+0000 7f71014ad700 1 Processor -- start 2024-04-25T11:01:01.696 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 -- start start 2024-04-25T11:01:01.696 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0084005120 con 0x7f009c12a6c0 2024-04-25T11:01:01.696 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f70fc090b70 0x7f70fc141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.696 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc122390 0x7f70fc1422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.697 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f0094014460 con 0x7f009c12a6c0 2024-04-25T11:01:01.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122c60 0x7f70fc11de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:01.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc08e3f0 con 0x7f70fc122390 2024-04-25T11:01:01.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc11f950 con 0x7f70fc122c60 2024-04-25T11:01:01.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f71014ad700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc13c9b0 con 0x7f70fc090b70 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f70fa7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc122390 0x7f70fc1422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f70fa7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc122390 0x7f70fc1422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34236/0 (socket says 172.21.15.155:34236) 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f70fa7fc700 1 -- 172.21.15.155:0/2416688683 learned_addr learned my addr 172.21.15.155:0/2416688683 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.689+0000 7f70faffd700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f70fc090b70 0x7f70fc141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70faffd700 1 -- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122c60 msgr2=0x7f70fc11de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70faffd700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122c60 0x7f70fc11de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70faffd700 1 -- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc122390 msgr2=0x7f70fc1422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70faffd700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc122390 0x7f70fc1422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70faffd700 1 -- 172.21.15.155:0/2416688683 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f4009a20 con 0x7f70fc090b70 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70faffd700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f70fc090b70 0x7f70fc141de0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f70fc0914d0 tx=0x7f70ec015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:01.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00d6b0 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00d810 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00dbd0 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec009250 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.690+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc1247b0 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.691+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f70ec027460 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.692+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f70ec056e50 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.693+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc121530 con 0x7f70fc090b70 2024-04-25T11:01:01.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.696+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f0094037a00 con 0x7f009c12a6c0 2024-04-25T11:01:01.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.696+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f009405b060 con 0x7f009c12a6c0 2024-04-25T11:01:01.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.698+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f70ec007880 con 0x7f70fc090b70 2024-04-25T11:01:01.807 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.807+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f139c11f020 con 0x7f139c127430 2024-04-25T11:01:01.808 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.807+0000 7f1392ffd700 1 -- 172.21.15.155:0/3670996381 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1394035090 con 0x7f139c127430 2024-04-25T11:01:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:38654705666 2024-04-25T11:01:01.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 >> v1:172.21.15.155:6816/43298 conn(0x7f1380041b80 legacy=0x7f1380044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c127430 msgr2=0x7f139c118f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c127430 0x7f139c118f00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1394014000 tx=0x7f1394009780).stop 2024-04-25T11:01:01.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 shutdown_connections 2024-04-25T11:01:01.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f139c127430 0x7f139c118f00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f139c067b10 0x7f139c134aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 --2- 172.21.15.155:0/3670996381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f139c124b50 0x7f139c134f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.809+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 >> 172.21.15.155:0/3670996381 conn(0x7f139c094420 msgr2=0x7f139c11c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.810+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 shutdown_connections 2024-04-25T11:01:01.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.810+0000 7f13a47a9700 1 -- 172.21.15.155:0/3670996381 wait complete. 2024-04-25T11:01:01.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.3 2024-04-25T11:01:01.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.844+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f735c10dc80 con 0x7f735c122030 2024-04-25T11:01:01.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.845+0000 7f734affd700 1 -- 172.21.15.155:0/1864426017 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f735c10dc80 con 0x7f735c122030 2024-04-25T11:01:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:42949672961 2024-04-25T11:01:01.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.845+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f00840057f0 con 0x7f009c12a6c0 2024-04-25T11:01:01.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f00927fc700 1 -- 172.21.15.155:0/3610371660 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f009405b030 con 0x7f009c12a6c0 2024-04-25T11:01:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:38654705666 2024-04-25T11:01:01.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 >> v1:172.21.15.155:6816/43298 conn(0x7f733c041b80 legacy=0x7f733c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 msgr2=0x7f735c132ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 0x7f735c132ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f734c00c510 tx=0x7f734c007c70).stop 2024-04-25T11:01:01.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 shutdown_connections 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f735c1278b0 0x7f735c10b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f735c122030 0x7f735c132ad0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 --2- 172.21.15.155:0/1864426017 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f735c122b20 0x7f735c132fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 >> 172.21.15.155:0/1864426017 conn(0x7f735c0956f0 msgr2=0x7f735c093ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 shutdown_connections 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.846+0000 7f7360813700 1 -- 172.21.15.155:0/1864426017 wait complete. 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.847+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f70fc047540 con 0x7f70fc090b70 2024-04-25T11:01:01.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.847+0000 7f70e37fe700 1 -- 172.21.15.155:0/2416688683 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70ec007880 con 0x7f70fc090b70 2024-04-25T11:01:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:42949672962 2024-04-25T11:01:01.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 >> v1:172.21.15.155:6816/43298 conn(0x7f0080041b80 legacy=0x7f0080044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 msgr2=0x7f009c1353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 0x7f009c1353f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0094009f10 tx=0x7f009400b040).stop 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 shutdown_connections 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f009c12a6c0 0x7f009c1353f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f009c0626c0 0x7f009c134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 --2- 172.21.15.155:0/3610371660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f009c067bc0 0x7f009c134f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 >> 172.21.15.155:0/3610371660 conn(0x7f009c0956f0 msgr2=0x7f009c065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 shutdown_connections 2024-04-25T11:01:01.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.848+0000 7f00a3b9d700 1 -- 172.21.15.155:0/3610371660 wait complete. 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.849+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 >> v1:172.21.15.155:6816/43298 conn(0x7f70e8041b80 legacy=0x7f70e8044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.849+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f70fc090b70 msgr2=0x7f70fc141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.849+0000 7f71014ad700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f70fc090b70 0x7f70fc141de0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f70fc0914d0 tx=0x7f70ec015000).stop 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 shutdown_connections 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70fc122c60 0x7f70fc11de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f70fc090b70 0x7f70fc141de0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 --2- 172.21.15.155:0/2416688683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc122390 0x7f70fc1422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:01.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 >> 172.21.15.155:0/2416688683 conn(0x7f70fc095700 msgr2=0x7f70fc11fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:01.852 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 shutdown_connections 2024-04-25T11:01:01.852 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:01.850+0000 7f71014ad700 1 -- 172.21.15.155:0/2416688683 wait complete. 2024-04-25T11:01:01.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672963 got 42949672962 for osd.2 2024-04-25T11:01:01.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672962 got 42949672961 for osd.1 2024-04-25T11:01:01.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.0 2024-04-25T11:01:02.827 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:01:02.865 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:01:02.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:01:02.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:01:02.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.974+0000 7f779ee27700 1 Processor -- start 2024-04-25T11:01:02.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.974+0000 7f779ee27700 1 -- start start 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.974+0000 7f779ee27700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798125410 0x7f77981257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.974+0000 7f779ee27700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7798125ce0 0x7f7798122090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779ee27700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 0x7f7798122a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779ee27700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980637d0 con 0x7f7798125ce0 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779ee27700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7798064c70 con 0x7f7798125410 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779ee27700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7798066f30 con 0x7f7798122580 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 0x7f7798122a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f7797fff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7798125ce0 0x7f7798122090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:02.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 0x7f7798122a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47256/0 (socket says 172.21.15.155:47256) 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 -- 172.21.15.155:0/2871140603 learned_addr learned my addr 172.21.15.155:0/2871140603 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779cbc3700 1 --2- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798125410 0x7f77981257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 -- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798125410 msgr2=0x7f77981257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 --2- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798125410 0x7f77981257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 -- 172.21.15.155:0/2871140603 >> v2:172.21.15.155:3300/0 conn(0x7f7798125ce0 msgr2=0x7f7798122090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 --2- 172.21.15.155:0/2871140603 >> v2:172.21.15.155:3300/0 conn(0x7f7798125ce0 0x7f7798122090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 -- 172.21.15.155:0/2871140603 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798067910 con 0x7f7798122580 2024-04-25T11:01:02.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f779d3c4700 1 --2- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 0x7f7798122a30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7790009f20 tx=0x7f7790009840).ready entity=mon.2 client_cookie=f499300d97e2996f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.975+0000 7f77977fe700 1 -- 172.21.15.155:0/2871140603 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000e070 con 0x7f7798122580 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f77977fe700 1 -- 172.21.15.155:0/2871140603 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779000b970 con 0x7f7798122580 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f77977fe700 1 -- 172.21.15.155:0/2871140603 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000bcf0 con 0x7f7798122580 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 -- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 msgr2=0x7f7798122a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 --2- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 0x7f7798122a30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7790009f20 tx=0x7f7790009840).stop 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 -- 172.21.15.155:0/2871140603 shutdown_connections 2024-04-25T11:01:02.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 --2- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798122580 0x7f7798122a30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 --2- 172.21.15.155:0/2871140603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798125410 0x7f77981257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 --2- 172.21.15.155:0/2871140603 >> v2:172.21.15.155:3300/0 conn(0x7f7798125ce0 0x7f7798122090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 -- 172.21.15.155:0/2871140603 >> 172.21.15.155:0/2871140603 conn(0x7f7798096b50 msgr2=0x7f7798091a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 -- 172.21.15.155:0/2871140603 shutdown_connections 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.976+0000 7f779ee27700 1 -- 172.21.15.155:0/2871140603 wait complete. 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.978+0000 7f779ee27700 1 Processor -- start 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.978+0000 7f779ee27700 1 -- start start 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.978+0000 7f779ee27700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798122580 0x7f7798134920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.978+0000 7f779ee27700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7798125410 0x7f7798134e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779ee27700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798125ce0 0x7f7798135300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779ee27700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798122120 con 0x7f7798125410 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779ee27700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7798118500 con 0x7f7798122580 2024-04-25T11:01:02.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779ee27700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77981357f0 con 0x7f7798125ce0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f7797fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7798125410 0x7f7798134e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798125ce0 0x7f7798135300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f7797fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7798125410 0x7f7798134e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34254/0 (socket says 172.21.15.155:34254) 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f7797fff700 1 -- 172.21.15.155:0/3053249483 learned_addr learned my addr 172.21.15.155:0/3053249483 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779cbc3700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798122580 0x7f7798134920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 -- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798122580 msgr2=0x7f7798134920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798122580 0x7f7798134920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 -- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7798125410 msgr2=0x7f7798134e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7798125410 0x7f7798134e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 -- 172.21.15.155:0/3053249483 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790009ae0 con 0x7f7798125ce0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.979+0000 7f779d3c4700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798125ce0 0x7f7798135300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f77980937b0 tx=0x7f7790009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.980+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790007170 con 0x7f7798125ce0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.980+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77900092e0 con 0x7f7798125ce0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.980+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779001e6a0 con 0x7f7798125ce0 2024-04-25T11:01:02.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.980+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7798117ce0 con 0x7f7798125ce0 2024-04-25T11:01:02.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.980+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981162c0 con 0x7f7798125ce0 2024-04-25T11:01:02.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.981+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7798047540 con 0x7f7798125ce0 2024-04-25T11:01:02.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.982+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f7790003040 con 0x7f7798125ce0 2024-04-25T11:01:02.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.982+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f7790056690 con 0x7f7798125ce0 2024-04-25T11:01:02.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:02.989+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f7790029be0 con 0x7f7798125ce0 2024-04-25T11:01:03.003 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 Processor -- start 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 -- start start 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000129fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 0x7f9000090880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9000090d70 0x7f9000130b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900006a290 con 0x7f9000090d70 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9000065f60 con 0x7f9000129bf0 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9006f61700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90001297a0 con 0x7f9000090460 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 0x7f9000090880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 0x7f9000090880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47278/0 (socket says 172.21.15.155:47278) 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f8ffffff700 1 -- 172.21.15.155:0/3405642440 learned_addr learned my addr 172.21.15.155:0/3405642440 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.003+0000 7f9004cfd700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000129fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.005 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8ffffff700 1 -- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 msgr2=0x7f9000129fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8ffffff700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000129fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8ffffff700 1 -- 172.21.15.155:0/3405642440 >> v2:172.21.15.155:3300/0 conn(0x7f9000090d70 msgr2=0x7f9000130b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8ffffff700 1 --2- 172.21.15.155:0/3405642440 >> v2:172.21.15.155:3300/0 conn(0x7f9000090d70 0x7f9000130b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8ffffff700 1 -- 172.21.15.155:0/3405642440 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000091220 con 0x7f9000090460 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f9004cfd700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8ffffff700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 0x7f9000090880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8fec006fd0 tx=0x7f8fec0090f0).ready entity=mon.2 client_cookie=48eef947dea53839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8fff7fe700 1 -- 172.21.15.155:0/3405642440 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00b580 con 0x7f9000090460 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8fff7fe700 1 -- 172.21.15.155:0/3405642440 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec00b6e0 con 0x7f9000090460 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.004+0000 7f8fff7fe700 1 -- 172.21.15.155:0/3405642440 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00ba80 con 0x7f9000090460 2024-04-25T11:01:03.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.005+0000 7f9006f61700 1 -- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 msgr2=0x7f9000090880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.005+0000 7f9006f61700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 0x7f9000090880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8fec006fd0 tx=0x7f8fec0090f0).stop 2024-04-25T11:01:03.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7fc448d36700 1 Processor -- start 2024-04-25T11:01:03.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7fc448d36700 1 -- start start 2024-04-25T11:01:03.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7f9006f61700 1 -- 172.21.15.155:0/3405642440 shutdown_connections 2024-04-25T11:01:03.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7f9006f61700 1 --2- 172.21.15.155:0/3405642440 >> v2:172.21.15.155:3300/0 conn(0x7f9000090d70 0x7f9000130b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:03.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7f9006f61700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7f9006f61700 1 --2- 172.21.15.155:0/3405642440 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090460 0x7f9000090880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7f9006f61700 1 -- 172.21.15.155:0/3405642440 >> 172.21.15.155:0/3405642440 conn(0x7f9000095700 msgr2=0x7f90000915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc448d36700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc444090f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc448d36700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc44412d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc448d36700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc444066ed0 con 0x7fc44405e140 2024-04-25T11:01:03.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc448d36700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440660e0 con 0x7fc44412acc0 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc448d36700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc444065e40 con 0x7fc444090b70 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc44412d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc441d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc444090f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc44412d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49296/0 (socket says 172.21.15.155:49296) 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc442d9d700 1 -- 172.21.15.155:0/2670136940 learned_addr learned my addr 172.21.15.155:0/2670136940 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc441d9b700 1 -- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 msgr2=0x7fc44412d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc441d9b700 1 --2- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc44412d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc441d9b700 1 -- 172.21.15.155:0/2670136940 >> v1:172.21.15.155:6789/0 conn(0x7fc44405e140 legacy=0x7fc444090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc441d9b700 1 -- 172.21.15.155:0/2670136940 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444065940 con 0x7fc444090b70 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc442d9d700 1 --2- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc44412d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc441d9b700 1 --2- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc444090f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc438009790 tx=0x7fc438009060).ready entity=mon.2 client_cookie=9568495b8db16bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.008+0000 7fc44159a700 1 -- 172.21.15.155:0/2670136940 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380090d0 con 0x7fc444090b70 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc44159a700 1 -- 172.21.15.155:0/2670136940 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43800f510 con 0x7fc444090b70 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc44159a700 1 -- 172.21.15.155:0/2670136940 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800f890 con 0x7fc444090b70 2024-04-25T11:01:03.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 -- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 msgr2=0x7fc444090f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 --2- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc444090f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc438009790 tx=0x7fc438009060).stop 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 -- 172.21.15.155:0/2670136940 shutdown_connections 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 --2- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc44412d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 --2- 172.21.15.155:0/2670136940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc444090f50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 -- 172.21.15.155:0/2670136940 >> 172.21.15.155:0/2670136940 conn(0x7fc444095700 msgr2=0x7fc4440915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 -- 172.21.15.155:0/2670136940 shutdown_connections 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.009+0000 7fc448d36700 1 -- 172.21.15.155:0/2670136940 wait complete. 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.007+0000 7f9006f61700 1 -- 172.21.15.155:0/3405642440 shutdown_connections 2024-04-25T11:01:03.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.010+0000 7f9006f61700 1 -- 172.21.15.155:0/3405642440 wait complete. 2024-04-25T11:01:03.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.010+0000 7fc448d36700 1 Processor -- start 2024-04-25T11:01:03.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 -- start start 2024-04-25T11:01:03.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7f9006f61700 1 Processor -- start 2024-04-25T11:01:03.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7f9006f61700 1 -- start start 2024-04-25T11:01:03.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc44405e140 0x7fc4441292d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc4441297c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc444129cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444088f50 con 0x7fc44405e140 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440885d0 con 0x7fc44412acc0 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc448d36700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc444089180 con 0x7fc444090b70 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc44259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc44405e140 0x7fc4441292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc44259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc44405e140 0x7fc4441292d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34256/0 (socket says 172.21.15.155:34256) 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc44259c700 1 -- 172.21.15.155:0/2882673525 learned_addr learned my addr 172.21.15.155:0/2882673525 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc441d9b700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc4441297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7f9006f61700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9000090460 0x7f9000126ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7f9006f61700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090d70 0x7f9000082fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7f9006f61700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000083490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7f9006f61700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900008e800 con 0x7f9000090460 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f9006f61700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9000084820 con 0x7f9000129bf0 2024-04-25T11:01:03.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f9006f61700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9000084f90 con 0x7f9000090d70 2024-04-25T11:01:03.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.011+0000 7fc442d9d700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc444129cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000083490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000083490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49302/0 (socket says 172.21.15.155:49302) 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 -- 172.21.15.155:0/1099510901 learned_addr learned my addr 172.21.15.155:0/1099510901 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f8ffffff700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090d70 0x7f9000082fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 -- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090d70 msgr2=0x7f9000082fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090d70 0x7f9000082fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 -- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9000090460 msgr2=0x7f9000126ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9000090460 0x7f9000126ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 -- 172.21.15.155:0/1099510901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009250 con 0x7f9000129bf0 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f90054fe700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000083490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8ff800b7e0 tx=0x7f8ff8009310).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc442d9d700 1 -- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 msgr2=0x7fc4441297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc442d9d700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc4441297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc442d9d700 1 -- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc44405e140 msgr2=0x7fc4441292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc442d9d700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc44405e140 0x7fc4441292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc442d9d700 1 -- 172.21.15.155:0/2882673525 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009ae0 con 0x7fc44412acc0 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc442d9d700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc444129cb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc43c009f10 tx=0x7fc43c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0092e0 con 0x7fc44412acc0 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c00f040 con 0x7fc44412acc0 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00cb70 con 0x7fc44412acc0 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800d5a0 con 0x7f9000129bf0 2024-04-25T11:01:03.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.013+0000 7f95b19b0700 1 Processor -- start 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.012+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4380092a0 con 0x7fc44412acc0 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.013+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc444087410 con 0x7fc44412acc0 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.013+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff800d740 con 0x7f9000129bf0 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.013+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800db00 con 0x7f9000129bf0 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.013+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec009370 con 0x7f9000129bf0 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.013+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90001300b0 con 0x7f9000129bf0 2024-04-25T11:01:03.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.014+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f8ff8019460 con 0x7f9000129bf0 2024-04-25T11:01:03.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.014+0000 7f95b19b0700 1 -- start start 2024-04-25T11:01:03.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.014+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f8ff8055210 con 0x7f9000129bf0 2024-04-25T11:01:03.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.014+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444047540 con 0x7fc44412acc0 2024-04-25T11:01:03.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.014+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7fc43c014460 con 0x7fc44412acc0 2024-04-25T11:01:03.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc43c055230 con 0x7fc44412acc0 2024-04-25T11:01:03.021 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95b19b0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f95ac069cf0 0x7f95ac127410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95b19b0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 0x7f95ac129cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95b19b0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0692a0 con 0x7f95ac1223b0 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95b19b0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac122190 con 0x7f95ac069cf0 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95b19b0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac122ea0 con 0x7f95ac127900 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 0x7f95ac129cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 0x7f95ac129cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49316/0 (socket says 172.21.15.155:49316) 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 -- 172.21.15.155:0/2780365430 learned_addr learned my addr 172.21.15.155:0/2780365430 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95aa7fc700 1 --2- 172.21.15.155:0/2780365430 >> v2:172.21.15.155:3300/0 conn(0x7f95ac069cf0 0x7f95ac127410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 -- 172.21.15.155:0/2780365430 >> v2:172.21.15.155:3300/0 conn(0x7f95ac069cf0 msgr2=0x7f95ac127410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 --2- 172.21.15.155:0/2780365430 >> v2:172.21.15.155:3300/0 conn(0x7f95ac069cf0 0x7f95ac127410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 -- 172.21.15.155:0/2780365430 >> v1:172.21.15.155:6789/0 conn(0x7f95ac1223b0 legacy=0x7f95ac122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 -- 172.21.15.155:0/2780365430 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac066350 con 0x7f95ac127900 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95aa7fc700 1 --2- 172.21.15.155:0/2780365430 >> v2:172.21.15.155:3300/0 conn(0x7f95ac069cf0 0x7f95ac127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:03.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.015+0000 7f95ab7fe700 1 --2- 172.21.15.155:0/2780365430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 0x7f95ac129cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f95a4014af0 tx=0x7f95a40143c0).ready entity=mon.1 client_cookie=4368a4c58891fd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:03.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.014+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9000047540 con 0x7f9000129bf0 2024-04-25T11:01:03.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.021+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7fc43c035bd0 con 0x7fc44412acc0 2024-04-25T11:01:03.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.028+0000 7f95a9ffb700 1 -- 172.21.15.155:0/2780365430 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4014430 con 0x7f95ac127900 2024-04-25T11:01:03.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.028+0000 7f95a9ffb700 1 -- 172.21.15.155:0/2780365430 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a4013b50 con 0x7f95ac127900 2024-04-25T11:01:03.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.028+0000 7f95a9ffb700 1 -- 172.21.15.155:0/2780365430 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4013ed0 con 0x7f95ac127900 2024-04-25T11:01:03.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.028+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f8ff8035d40 con 0x7f9000129bf0 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 -- 172.21.15.155:0/2780365430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 msgr2=0x7f95ac129cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 --2- 172.21.15.155:0/2780365430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 0x7f95ac129cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f95a4014af0 tx=0x7f95a40143c0).stop 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 -- 172.21.15.155:0/2780365430 shutdown_connections 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 --2- 172.21.15.155:0/2780365430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac127900 0x7f95ac129cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 --2- 172.21.15.155:0/2780365430 >> v2:172.21.15.155:3300/0 conn(0x7f95ac069cf0 0x7f95ac127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 -- 172.21.15.155:0/2780365430 >> 172.21.15.155:0/2780365430 conn(0x7f95ac094260 msgr2=0x7f95ac094670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 -- 172.21.15.155:0/2780365430 shutdown_connections 2024-04-25T11:01:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.029+0000 7f95b19b0700 1 -- 172.21.15.155:0/2780365430 wait complete. 2024-04-25T11:01:03.031 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.031+0000 7f95b19b0700 1 Processor -- start 2024-04-25T11:01:03.031 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.031+0000 7f95b19b0700 1 -- start start 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95b19b0700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 0x7f95ac1240b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95b19b0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac1223b0 0x7f95ac1245a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95b19b0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac127900 0x7f95ac124a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95b19b0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac119030 con 0x7f95ac127900 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95b19b0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac10b2d0 con 0x7f95ac1223b0 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95b19b0700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac126e30 con 0x7f95ac069cf0 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 0x7f95ac1240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 0x7f95ac1240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47322/0 (socket says 172.21.15.155:47322) 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 -- 172.21.15.155:0/570897463 learned_addr learned my addr 172.21.15.155:0/570897463 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aa7fc700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac1223b0 0x7f95ac1245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95ab7fe700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac127900 0x7f95ac124a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 -- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac1223b0 msgr2=0x7f95ac1245a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac1223b0 0x7f95ac1245a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 -- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac127900 msgr2=0x7f95ac124a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac127900 0x7f95ac124a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 -- 172.21.15.155:0/570897463 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a4014600 con 0x7f95ac069cf0 2024-04-25T11:01:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95aaffd700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 0x7f95ac1240b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f959c00b4a0 tx=0x7f959c006f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.032+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00be10 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.033+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c012460 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.033+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac125be0 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.033+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c012820 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.033+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1251c0 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.034+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f959c01c3f0 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.034+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac1253b0 con 0x7f95ac069cf0 2024-04-25T11:01:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.034+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f959c0547a0 con 0x7f95ac069cf0 2024-04-25T11:01:03.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.038+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f959c035140 con 0x7f95ac069cf0 2024-04-25T11:01:03.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.153+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7798123490 con 0x7f7798125ce0 2024-04-25T11:01:03.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.154+0000 7f77957fa700 1 -- 172.21.15.155:0/3053249483 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7790023090 con 0x7f7798125ce0 2024-04-25T11:01:03.155 INFO:teuthology.orchestra.run.smithi155.stdout:38654705666 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 >> v1:172.21.15.155:6816/43298 conn(0x7f777c041bd0 legacy=0x7f777c044090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798125ce0 msgr2=0x7f7798135300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798125ce0 0x7f7798135300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f77980937b0 tx=0x7f7790009840).stop 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 shutdown_connections 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7798125ce0 0x7f7798135300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7798122580 0x7f7798134920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 --2- 172.21.15.155:0/3053249483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7798125410 0x7f7798134e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 >> 172.21.15.155:0/3053249483 conn(0x7f7798096b50 msgr2=0x7f7798123750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 shutdown_connections 2024-04-25T11:01:03.157 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.156+0000 7f779ee27700 1 -- 172.21.15.155:0/3053249483 wait complete. 2024-04-25T11:01:03.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.3 2024-04-25T11:01:03.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.192+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95ac121ec0 con 0x7f95ac069cf0 2024-04-25T11:01:03.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.192+0000 7f95937fe700 1 -- 172.21.15.155:0/570897463 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f959c038930 con 0x7f95ac069cf0 2024-04-25T11:01:03.193 INFO:teuthology.orchestra.run.smithi155.stdout:38654705666 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 >> v1:172.21.15.155:6816/43298 conn(0x7f9598041b80 legacy=0x7f9598044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 msgr2=0x7f95ac1240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 0x7f95ac1240b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f959c00b4a0 tx=0x7f959c006f90).stop 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 shutdown_connections 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac127900 0x7f95ac124a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f95ac069cf0 0x7f95ac1240b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 --2- 172.21.15.155:0/570897463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95ac1223b0 0x7f95ac1245a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 >> 172.21.15.155:0/570897463 conn(0x7f95ac094260 msgr2=0x7f95ac119720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 shutdown_connections 2024-04-25T11:01:03.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.194+0000 7f95b19b0700 1 -- 172.21.15.155:0/570897463 wait complete. 2024-04-25T11:01:03.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.0 2024-04-25T11:01:03.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.213+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc44412c2a0 con 0x7fc44412acc0 2024-04-25T11:01:03.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.213+0000 7fc432ffd700 1 -- 172.21.15.155:0/2882673525 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc43c0393c0 con 0x7fc44412acc0 2024-04-25T11:01:03.214 INFO:teuthology.orchestra.run.smithi155.stdout:42949672961 2024-04-25T11:01:03.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 >> v1:172.21.15.155:6816/43298 conn(0x7fc424041b80 legacy=0x7fc424044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 msgr2=0x7fc444129cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc444129cb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc43c009f10 tx=0x7fc43c00b040).stop 2024-04-25T11:01:03.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 shutdown_connections 2024-04-25T11:01:03.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc44412acc0 0x7fc444129cb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc44405e140 0x7fc4441292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 --2- 172.21.15.155:0/2882673525 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc444090b70 0x7fc4441297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 >> 172.21.15.155:0/2882673525 conn(0x7fc444095700 msgr2=0x7fc444092a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 shutdown_connections 2024-04-25T11:01:03.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.215+0000 7fc448d36700 1 -- 172.21.15.155:0/2882673525 wait complete. 2024-04-25T11:01:03.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.218+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9000083da0 con 0x7f9000129bf0 2024-04-25T11:01:03.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.218+0000 7f8ffd7fa700 1 -- 172.21.15.155:0/1099510901 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ff8039360 con 0x7f9000129bf0 2024-04-25T11:01:03.219 INFO:teuthology.orchestra.run.smithi155.stdout:42949672962 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 >> v1:172.21.15.155:6816/43298 conn(0x7f8fe4041bd0 legacy=0x7f8fe4044090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 msgr2=0x7f9000083490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000083490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8ff800b7e0 tx=0x7f8ff8009310).stop 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 shutdown_connections 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9000129bf0 0x7f9000083490 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9000090460 0x7f9000126ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 --2- 172.21.15.155:0/1099510901 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f9000090d70 0x7f9000082fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 >> 172.21.15.155:0/1099510901 conn(0x7f9000095700 msgr2=0x7f90000691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 shutdown_connections 2024-04-25T11:01:03.221 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:03.220+0000 7f9006f61700 1 -- 172.21.15.155:0/1099510901 wait complete. 2024-04-25T11:01:03.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672962 got 42949672961 for osd.1 2024-04-25T11:01:03.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672963 got 42949672962 for osd.2 2024-04-25T11:01:04.173 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:01:04.208 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:01:04.229 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:01:04.233 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:01:04.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.317+0000 7f21809d6700 1 Processor -- start 2024-04-25T11:01:04.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.317+0000 7f21809d6700 1 -- start start 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f21809d6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c122410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f21809d6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f217c122900 0x7f217c11ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f21809d6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c069b40 con 0x7f217c11f0c0 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f21809d6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c069020 con 0x7f217c122900 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f21809d6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c068890 con 0x7f217c122030 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217ad9d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f217c11f0c0 0x7f217c1293d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44688/0 (socket says 172.21.15.155:44688) 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217ad9d700 1 -- 172.21.15.155:0/2788257797 learned_addr learned my addr 172.21.15.155:0/2788257797 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217a59c700 1 --2- 172.21.15.155:0/2788257797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f2179d9b700 1 --2- 172.21.15.155:0/2788257797 >> v2:172.21.15.155:3300/0 conn(0x7f217c122900 0x7f217c11ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217959a700 1 -- 172.21.15.155:0/2788257797 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952294444 0 0) 0x7f217c069b40 con 0x7f217c11f0c0 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217959a700 1 -- 172.21.15.155:0/2788257797 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21640033f0 con 0x7f217c11f0c0 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217a59c700 1 -- 172.21.15.155:0/2788257797 >> v2:172.21.15.155:3300/0 conn(0x7f217c122900 msgr2=0x7f217c11ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.318+0000 7f217a59c700 1 --2- 172.21.15.155:0/2788257797 >> v2:172.21.15.155:3300/0 conn(0x7f217c122900 0x7f217c11ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.319+0000 7f217a59c700 1 -- 172.21.15.155:0/2788257797 >> v1:172.21.15.155:6789/0 conn(0x7f217c11f0c0 legacy=0x7f217c1293d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.319+0000 7f217a59c700 1 -- 172.21.15.155:0/2788257797 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c069880 con 0x7f217c122030 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.319+0000 7f217a59c700 1 --2- 172.21.15.155:0/2788257797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c122410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f216c00ba00 tx=0x7f216c00b300).ready entity=mon.1 client_cookie=c0f227a1304a2df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.319+0000 7f217959a700 1 -- 172.21.15.155:0/2788257797 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00d8c0 con 0x7f217c122030 2024-04-25T11:01:04.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.319+0000 7f217959a700 1 -- 172.21.15.155:0/2788257797 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c00da20 con 0x7f217c122030 2024-04-25T11:01:04.321 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.319+0000 7f217959a700 1 -- 172.21.15.155:0/2788257797 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00dda0 con 0x7f217c122030 2024-04-25T11:01:04.321 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 -- 172.21.15.155:0/2788257797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 msgr2=0x7f217c122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.321 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 --2- 172.21.15.155:0/2788257797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c122410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f216c00ba00 tx=0x7f216c00b300).stop 2024-04-25T11:01:04.321 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 -- 172.21.15.155:0/2788257797 shutdown_connections 2024-04-25T11:01:04.321 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 --2- 172.21.15.155:0/2788257797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c122410 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 --2- 172.21.15.155:0/2788257797 >> v2:172.21.15.155:3300/0 conn(0x7f217c122900 0x7f217c11ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 -- 172.21.15.155:0/2788257797 >> 172.21.15.155:0/2788257797 conn(0x7f217c0956f0 msgr2=0x7f217c095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 -- 172.21.15.155:0/2788257797 shutdown_connections 2024-04-25T11:01:04.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.320+0000 7f21809d6700 1 -- 172.21.15.155:0/2788257797 wait complete. 2024-04-25T11:01:04.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.321+0000 7f21809d6700 1 Processor -- start 2024-04-25T11:01:04.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 -- start start 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 0x7f217c1154d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c1159c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f217c122900 0x7f217c115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1189d0 con 0x7f217c11f0c0 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c094430 con 0x7f217c122030 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f21809d6700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f217c116e90 con 0x7f217c122900 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f217a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 0x7f217c1154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f217a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 0x7f217c1154d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34294/0 (socket says 172.21.15.155:34294) 2024-04-25T11:01:04.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f217a59c700 1 -- 172.21.15.155:0/2085765554 learned_addr learned my addr 172.21.15.155:0/2085765554 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.322+0000 7f2179d9b700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c1159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f217ad9d700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f217c122900 0x7f217c115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f2179d9b700 1 -- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f217c122900 msgr2=0x7f217c115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f2179d9b700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f217c122900 0x7f217c115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f2179d9b700 1 -- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 msgr2=0x7f217c1154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f2179d9b700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 0x7f217c1154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f2179d9b700 1 -- 172.21.15.155:0/2085765554 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c00b540 con 0x7f217c122030 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f217a59c700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 0x7f217c1154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f2179d9b700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c1159c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f217000e900 tx=0x7f2170009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170009470 con 0x7f217c122030 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170012600 con 0x7f217c122030 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c1163a0 con 0x7f217c122030 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170012980 con 0x7f217c122030 2024-04-25T11:01:04.325 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.323+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c13b230 con 0x7f217c122030 2024-04-25T11:01:04.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.325+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f2170012d30 con 0x7f217c122030 2024-04-25T11:01:04.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.325+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f2170055450 con 0x7f217c122030 2024-04-25T11:01:04.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.325+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c062e80 con 0x7f217c122030 2024-04-25T11:01:04.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.329+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f21700360c0 con 0x7f217c122030 2024-04-25T11:01:04.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.364+0000 7fd6fd8a7700 1 Processor -- start 2024-04-25T11:01:04.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.364+0000 7fd6fd8a7700 1 -- start start 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6fd8a7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123490 0x7fd6f8123870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6fd8a7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6f8090b70 0x7fd6f812b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6fd8a7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f8066360 con 0x7fd6f8123d60 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6fd8a7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f8068ed0 con 0x7fd6f8090b70 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6fd8a7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8065a40 con 0x7fd6f8123490 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6f8090b70 0x7fd6f812b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6f8090b70 0x7fd6f812b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34306/0 (socket says 172.21.15.155:34306) 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f67fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd6f8123d60 0x7fd6f8090460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44702/0 (socket says 172.21.15.155:44702) 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 -- 172.21.15.155:0/2441598146 learned_addr learned my addr 172.21.15.155:0/2441598146 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 -- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123490 msgr2=0x7fd6f8123870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 --2- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123490 0x7fd6f8123870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 -- 172.21.15.155:0/2441598146 >> v1:172.21.15.155:6789/0 conn(0x7fd6f8123d60 legacy=0x7fd6f8090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 -- 172.21.15.155:0/2441598146 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8069110 con 0x7fd6f8090b70 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f5ffb700 1 -- 172.21.15.155:0/2441598146 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752241711 0 0) 0x7fd6f8066360 con 0x7fd6f8123d60 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.365+0000 7fd6f77fe700 1 --2- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8090b70 0x7fd6f812b540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd6f0009f50 tx=0x7fd6f0009840).ready entity=mon.0 client_cookie=8421acaced1a8270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6f5ffb700 1 -- 172.21.15.155:0/2441598146 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000e070 con 0x7fd6f8090b70 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6f5ffb700 1 -- 172.21.15.155:0/2441598146 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000ba70 con 0x7fd6f8090b70 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6f5ffb700 1 -- 172.21.15.155:0/2441598146 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000bdf0 con 0x7fd6f8090b70 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8090b70 msgr2=0x7fd6f812b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8090b70 0x7fd6f812b540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd6f0009f50 tx=0x7fd6f0009840).stop 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/2441598146 shutdown_connections 2024-04-25T11:01:04.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8090b70 0x7fd6f812b540 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/2441598146 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123490 0x7fd6f8123870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/2441598146 >> 172.21.15.155:0/2441598146 conn(0x7fd6f8095700 msgr2=0x7fd6f80915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/2441598146 shutdown_connections 2024-04-25T11:01:04.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.366+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/2441598146 wait complete. 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.368+0000 7fd6fd8a7700 1 Processor -- start 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.368+0000 7fd6fd8a7700 1 -- start start 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.368+0000 7fd6fd8a7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6f8090b70 0x7fd6f811dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6fd8a7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 0x7fd6f811e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6fd8a7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123d60 0x7fd6f811e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6fd8a7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f811ee80 con 0x7fd6f8123490 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6fd8a7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f811f0d0 con 0x7fd6f8090b70 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6fd8a7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8083980 con 0x7fd6f8123d60 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f67fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 0x7fd6f811e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6f8090b70 0x7fd6f811dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f67fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 0x7fd6f811e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34312/0 (socket says 172.21.15.155:34312) 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f67fc700 1 -- 172.21.15.155:0/1052027597 learned_addr learned my addr 172.21.15.155:0/1052027597 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 -- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123d60 msgr2=0x7fd6f811e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123d60 0x7fd6f811e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 -- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 msgr2=0x7fd6f811e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 0x7fd6f811e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 -- 172.21.15.155:0/1052027597 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0009ae0 con 0x7fd6f8090b70 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f67fc700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 0x7fd6f811e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.369+0000 7fd6f6ffd700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6f8090b70 0x7fd6f811dfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd6e800b460 tx=0x7fd6e8009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.370+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80090d0 con 0x7fd6f8090b70 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.370+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e8002320 con 0x7fd6f8090b70 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.370+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80835b0 con 0x7fd6f8090b70 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.370+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80026e0 con 0x7fd6f8090b70 2024-04-25T11:01:04.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.370+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f8132ad0 con 0x7fd6f8090b70 2024-04-25T11:01:04.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.371+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f808f530 con 0x7fd6f8090b70 2024-04-25T11:01:04.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.372+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7fd6e8002ad0 con 0x7fd6f8090b70 2024-04-25T11:01:04.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.372+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fd6e8054fa0 con 0x7fd6f8090b70 2024-04-25T11:01:04.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.376+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7fd6e80358e0 con 0x7fd6f8090b70 2024-04-25T11:01:04.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.387+0000 7f1c735cf700 1 Processor -- start 2024-04-25T11:01:04.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.387+0000 7f1c735cf700 1 -- start start 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c735cf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c090460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c735cf700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 0x7f1c6c090d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c735cf700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c066ed0 con 0x7f1c6c129cb0 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c735cf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0660e0 con 0x7f1c6c05e140 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c735cf700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c065e40 con 0x7f1c6c090950 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 0x7f1c6c090d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 0x7f1c6c090d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47364/0 (socket says 172.21.15.155:47364) 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 -- 172.21.15.155:0/3574711517 learned_addr learned my addr 172.21.15.155:0/3574711517 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c7136b700 1 --2- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c090460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 -- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 msgr2=0x7f1c6c090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 --2- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c090460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 -- 172.21.15.155:0/3574711517 >> v1:172.21.15.155:6789/0 conn(0x7f1c6c129cb0 legacy=0x7f1c6c12c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 -- 172.21.15.155:0/3574711517 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c065940 con 0x7f1c6c090950 2024-04-25T11:01:04.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.388+0000 7f1c70b6a700 1 --2- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 0x7f1c6c090d30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1c5c009830 tx=0x7f1c5c0090f0).ready entity=mon.2 client_cookie=a2b10ee1a1ce65c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c63fff700 1 -- 172.21.15.155:0/3574711517 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00b640 con 0x7f1c6c090950 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c63fff700 1 -- 172.21.15.155:0/3574711517 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c00b7a0 con 0x7f1c6c090950 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c63fff700 1 -- 172.21.15.155:0/3574711517 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00bb40 con 0x7f1c6c090950 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 -- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 msgr2=0x7f1c6c090d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 --2- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 0x7f1c6c090d30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1c5c009830 tx=0x7f1c5c0090f0).stop 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 -- 172.21.15.155:0/3574711517 shutdown_connections 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 --2- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 --2- 172.21.15.155:0/3574711517 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c090950 0x7f1c6c090d30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 -- 172.21.15.155:0/3574711517 >> 172.21.15.155:0/3574711517 conn(0x7f1c6c095700 msgr2=0x7f1c6c0915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 -- 172.21.15.155:0/3574711517 shutdown_connections 2024-04-25T11:01:04.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.389+0000 7f1c735cf700 1 -- 172.21.15.155:0/3574711517 wait complete. 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.390+0000 7f1c735cf700 1 Processor -- start 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 -- start start 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c0831e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c6c090950 0x7f1c6c0836d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c129cb0 0x7f1c6c083bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c08dee0 con 0x7f1c6c090950 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0840b0 con 0x7f1c6c05e140 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c735cf700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c084300 con 0x7f1c6c129cb0 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c7136b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c0831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c7136b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c0831e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49388/0 (socket says 172.21.15.155:49388) 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c7136b700 1 -- 172.21.15.155:0/1100023632 learned_addr learned my addr 172.21.15.155:0/1100023632 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.391+0000 7f1c71b6c700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c129cb0 0x7f1c6c083bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c70b6a700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c6c090950 0x7f1c6c0836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c71b6c700 1 -- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 msgr2=0x7f1c6c0831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c71b6c700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c0831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c71b6c700 1 -- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c6c090950 msgr2=0x7f1c6c0836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c71b6c700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c6c090950 0x7f1c6c0836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c71b6c700 1 -- 172.21.15.155:0/1100023632 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009370 con 0x7f1c6c129cb0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c71b6c700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c129cb0 0x7f1c6c083bc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1c6c05e590 tx=0x7f1c64001530).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400e4a0 con 0x7f1c6c129cb0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6400e600 con 0x7f1c6c129cb0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400e980 con 0x7f1c6c129cb0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c086620 con 0x7f1c6c129cb0 2024-04-25T11:01:04.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.392+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c1394d0 con 0x7f1c6c129cb0 2024-04-25T11:01:04.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.394+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c069f90 con 0x7f1c6c129cb0 2024-04-25T11:01:04.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.394+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f1c640123f0 con 0x7f1c6c129cb0 2024-04-25T11:01:04.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.394+0000 7f4fb0809700 1 Processor -- start 2024-04-25T11:01:04.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.394+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f1c64054970 con 0x7f1c6c129cb0 2024-04-25T11:01:04.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.394+0000 7f4fb0809700 1 -- start start 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fb0809700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4fac122030 0x7f4fac122410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fb0809700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122900 0x7f4fac11ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fb0809700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fb0809700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac069020 con 0x7f4fac122030 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fb0809700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac11fa60 con 0x7f4fac122900 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fb0809700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac069b40 con 0x7f4fac11f0c0 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4faa59c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4fac122030 0x7f4fac122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4faa59c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4fac122030 0x7f4fac122410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34332/0 (socket says 172.21.15.155:34332) 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4faa59c700 1 -- 172.21.15.155:0/56096466 learned_addr learned my addr 172.21.15.155:0/56096466 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.395+0000 7f4fa9d9b700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122900 0x7f4fac11ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4faa59c700 1 -- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 msgr2=0x7f4fac11f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4faa59c700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4faa59c700 1 -- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122900 msgr2=0x7f4fac11ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:04.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4faa59c700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122900 0x7f4fac11ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4faa59c700 1 -- 172.21.15.155:0/56096466 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac068890 con 0x7f4fac122030 2024-04-25T11:01:04.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4faa59c700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122030 0x7f4fac122410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f4f9c006fd0 tx=0x7f4f9c00b3b0).ready entity=mon.0 client_cookie=ebff3d9e5453af87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4fa959a700 1 -- 172.21.15.155:0/56096466 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00db40 con 0x7f4fac122030 2024-04-25T11:01:04.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4fa959a700 1 -- 172.21.15.155:0/56096466 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00dca0 con 0x7f4fac122030 2024-04-25T11:01:04.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.396+0000 7f4fa959a700 1 -- 172.21.15.155:0/56096466 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c010680 con 0x7f4fac122030 2024-04-25T11:01:04.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 -- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122030 msgr2=0x7f4fac122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122030 0x7f4fac122410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f4f9c006fd0 tx=0x7f4f9c00b3b0).stop 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 -- 172.21.15.155:0/56096466 shutdown_connections 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122030 0x7f4fac122410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 --2- 172.21.15.155:0/56096466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122900 0x7f4fac11ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 -- 172.21.15.155:0/56096466 >> 172.21.15.155:0/56096466 conn(0x7f4fac0956f0 msgr2=0x7f4fac095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 -- 172.21.15.155:0/56096466 shutdown_connections 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.397+0000 7f4fb0809700 1 -- 172.21.15.155:0/56096466 wait complete. 2024-04-25T11:01:04.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 Processor -- start 2024-04-25T11:01:04.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f1c64035580 con 0x7f1c6c129cb0 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 -- start start 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122030 0x7f4fac11bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122900 0x7f4fac11bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac11c4c0 con 0x7f4fac122900 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1101d0 con 0x7f4fac122030 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.399+0000 7f4fb0809700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1109b0 con 0x7f4fac11f0c0 2024-04-25T11:01:04.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47388/0 (socket says 172.21.15.155:47388) 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 -- 172.21.15.155:0/1620753331 learned_addr learned my addr 172.21.15.155:0/1620753331 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faad9d700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122900 0x7f4fac11bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 -- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122030 msgr2=0x7f4fac11bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122030 0x7f4fac11bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 -- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122900 msgr2=0x7f4fac11bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122900 0x7f4fac11bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 -- 172.21.15.155:0/1620753331 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c00b610 con 0x7f4fac11f0c0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faad9d700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122900 0x7f4fac11bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.400+0000 7f4faa59c700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11b5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f4f9c00bfd0 tx=0x7f4fac068cd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.401+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c010640 con 0x7f4fac11f0c0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.401+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac110410 con 0x7f4fac11f0c0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.401+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac111200 con 0x7f4fac11f0c0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.401+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c019870 con 0x7f4fac11f0c0 2024-04-25T11:01:04.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.401+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c019a00 con 0x7f4fac11f0c0 2024-04-25T11:01:04.404 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.402+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f4f9c00f3d0 con 0x7f4fac11f0c0 2024-04-25T11:01:04.404 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.402+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac047540 con 0x7f4fac11f0c0 2024-04-25T11:01:04.404 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.402+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f4f9c056ee0 con 0x7f4fac11f0c0 2024-04-25T11:01:04.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.410+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f4f9c007cf0 con 0x7f4fac11f0c0 2024-04-25T11:01:04.502 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.502+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f217c047540 con 0x7f217c122030 2024-04-25T11:01:04.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.502+0000 7f216affd700 1 -- 172.21.15.155:0/2085765554 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21700396e0 con 0x7f217c122030 2024-04-25T11:01:04.503 INFO:teuthology.orchestra.run.smithi155.stdout:38654705667 2024-04-25T11:01:04.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 >> v1:172.21.15.155:6816/43298 conn(0x7f215c041b80 legacy=0x7f215c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 msgr2=0x7f217c1159c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c1159c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f217000e900 tx=0x7f2170009060).stop 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 shutdown_connections 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f217c122900 0x7f217c115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f217c11f0c0 0x7f217c1154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 --2- 172.21.15.155:0/2085765554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f217c122030 0x7f217c1159c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 >> 172.21.15.155:0/2085765554 conn(0x7f217c0956f0 msgr2=0x7f217c11e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 shutdown_connections 2024-04-25T11:01:04.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.504+0000 7f21809d6700 1 -- 172.21.15.155:0/2085765554 wait complete. 2024-04-25T11:01:04.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.3 2024-04-25T11:01:04.516 DEBUG:teuthology.parallel:result is None 2024-04-25T11:01:04.532 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.532+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd6f8047540 con 0x7fd6f8090b70 2024-04-25T11:01:04.533 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.532+0000 7fd6df7fe700 1 -- 172.21.15.155:0/1052027597 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd6e80390d0 con 0x7fd6f8090b70 2024-04-25T11:01:04.533 INFO:teuthology.orchestra.run.smithi155.stdout:38654705667 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 >> v1:172.21.15.155:6816/43298 conn(0x7fd6e4041b80 legacy=0x7fd6e4044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6f8090b70 msgr2=0x7fd6f811dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6f8090b70 0x7fd6f811dfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd6e800b460 tx=0x7fd6e8009060).stop 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 shutdown_connections 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd6f8123d60 0x7fd6f811e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6f8090b70 0x7fd6f811dfb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 --2- 172.21.15.155:0/1052027597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6f8123490 0x7fd6f811e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 >> 172.21.15.155:0/1052027597 conn(0x7fd6f8095700 msgr2=0x7fd6f80928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 shutdown_connections 2024-04-25T11:01:04.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.534+0000 7fd6fd8a7700 1 -- 172.21.15.155:0/1052027597 wait complete. 2024-04-25T11:01:04.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.0 2024-04-25T11:01:04.547 DEBUG:teuthology.parallel:result is None 2024-04-25T11:01:04.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.554+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1c6c086060 con 0x7f1c6c129cb0 2024-04-25T11:01:04.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.554+0000 7f1c61ffb700 1 -- 172.21.15.155:0/1100023632 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c64038ba0 con 0x7f1c6c129cb0 2024-04-25T11:01:04.555 INFO:teuthology.orchestra.run.smithi155.stdout:42949672962 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 >> v1:172.21.15.155:6816/43298 conn(0x7f1c50041bd0 legacy=0x7f1c50044090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c129cb0 msgr2=0x7f1c6c083bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c129cb0 0x7f1c6c083bc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1c6c05e590 tx=0x7f1c64001530).stop 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 shutdown_connections 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c6c129cb0 0x7f1c6c083bc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c6c05e140 0x7f1c6c0831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 --2- 172.21.15.155:0/1100023632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c6c090950 0x7f1c6c0836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 >> 172.21.15.155:0/1100023632 conn(0x7f1c6c095700 msgr2=0x7f1c6c093740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 shutdown_connections 2024-04-25T11:01:04.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.556+0000 7f1c735cf700 1 -- 172.21.15.155:0/1100023632 wait complete. 2024-04-25T11:01:04.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.560+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4fac0645d0 con 0x7f4fac11f0c0 2024-04-25T11:01:04.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.561+0000 7f4f9affd700 1 -- 172.21.15.155:0/1620753331 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f9c03b070 con 0x7f4fac11f0c0 2024-04-25T11:01:04.561 INFO:teuthology.orchestra.run.smithi155.stdout:42949672963 2024-04-25T11:01:04.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 >> v1:172.21.15.155:6816/43298 conn(0x7f4f8c041b80 legacy=0x7f4f8c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 msgr2=0x7f4fac11b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11b5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f4f9c00bfd0 tx=0x7f4fac068cd0).stop 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 shutdown_connections 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fac122900 0x7f4fac11bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4fac11f0c0 0x7f4fac11b5f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 --2- 172.21.15.155:0/1620753331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fac122030 0x7f4fac11bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 >> 172.21.15.155:0/1620753331 conn(0x7f4fac0956f0 msgr2=0x7f4fac11df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 shutdown_connections 2024-04-25T11:01:04.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.563+0000 7f4fb0809700 1 -- 172.21.15.155:0/1620753331 wait complete. 2024-04-25T11:01:04.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672962 got 42949672962 for osd.1 2024-04-25T11:01:04.570 DEBUG:teuthology.parallel:result is None 2024-04-25T11:01:04.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672963 got 42949672963 for osd.2 2024-04-25T11:01:04.576 DEBUG:teuthology.parallel:result is None 2024-04-25T11:01:04.576 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:01:04.576 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.730+0000 7f8166fd7700 1 Processor -- start 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.730+0000 7f8166fd7700 1 -- start start 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f8166fd7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 0x7f81601284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f8166fd7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81601289c0 0x7f816012adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f8166fd7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81601232c0 con 0x7f81601234e0 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f8166fd7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81600693a0 con 0x7f81601289c0 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f8166fd7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8160123fd0 con 0x7f8160069dd0 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 0x7f81601284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f8165574700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81601289c0 0x7f816012adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 0x7f81601284d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49410/0 (socket says 172.21.15.155:49410) 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 -- 172.21.15.155:0/3285280594 learned_addr learned my addr 172.21.15.155:0/3285280594 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 -- 172.21.15.155:0/3285280594 >> v2:172.21.15.155:3300/0 conn(0x7f81601289c0 msgr2=0x7f816012adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 --2- 172.21.15.155:0/3285280594 >> v2:172.21.15.155:3300/0 conn(0x7f81601289c0 0x7f816012adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 -- 172.21.15.155:0/3285280594 >> v1:172.21.15.155:6789/0 conn(0x7f81601234e0 legacy=0x7f81601238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 -- 172.21.15.155:0/3285280594 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160062880 con 0x7f8160069dd0 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.731+0000 7f815ffff700 1 --2- 172.21.15.155:0/3285280594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 0x7f81601284d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f814c009830 tx=0x7f814c0090f0).ready entity=mon.1 client_cookie=372162eb054594c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f815f7fe700 1 -- 172.21.15.155:0/3285280594 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00b580 con 0x7f8160069dd0 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f815f7fe700 1 -- 172.21.15.155:0/3285280594 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c00b6e0 con 0x7f8160069dd0 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f815f7fe700 1 -- 172.21.15.155:0/3285280594 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00ba80 con 0x7f8160069dd0 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 -- 172.21.15.155:0/3285280594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 msgr2=0x7f81601284d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 --2- 172.21.15.155:0/3285280594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 0x7f81601284d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f814c009830 tx=0x7f814c0090f0).stop 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 -- 172.21.15.155:0/3285280594 shutdown_connections 2024-04-25T11:01:04.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 --2- 172.21.15.155:0/3285280594 >> v2:172.21.15.155:3300/0 conn(0x7f81601289c0 0x7f816012adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 --2- 172.21.15.155:0/3285280594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8160069dd0 0x7f81601284d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 -- 172.21.15.155:0/3285280594 >> 172.21.15.155:0/3285280594 conn(0x7f81600956f0 msgr2=0x7f8160095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 -- 172.21.15.155:0/3285280594 shutdown_connections 2024-04-25T11:01:04.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.732+0000 7f8166fd7700 1 -- 172.21.15.155:0/3285280594 wait complete. 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.733+0000 7f8166fd7700 1 Processor -- start 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.733+0000 7f8166fd7700 1 -- start start 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 0x7f81601242a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81601234e0 0x7f8160124790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81601289c0 0x7f8160124c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81601268a0 con 0x7f8160069dd0 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8160126ae0 con 0x7f81601234e0 2024-04-25T11:01:04.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81601274c0 con 0x7f81601289c0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 0x7f81601242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 0x7f81601242a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34360/0 (socket says 172.21.15.155:34360) 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 -- 172.21.15.155:0/470834721 learned_addr learned my addr 172.21.15.155:0/470834721 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f815ffff700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81601234e0 0x7f8160124790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8165574700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81601289c0 0x7f8160124c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 -- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81601289c0 msgr2=0x7f8160124c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81601289c0 0x7f8160124c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 -- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81601234e0 msgr2=0x7f8160124790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81601234e0 0x7f8160124790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 -- 172.21.15.155:0/470834721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c009370 con 0x7f8160069dd0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f815ffff700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81601234e0 0x7f8160124790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8164d73700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 0x7f81601242a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f815400c4c0 tx=0x7f8154007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815400ce10 con 0x7f8160069dd0 2024-04-25T11:01:04.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8154001e90 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8160127080 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154007270 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.734+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81601328c0 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.735+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f81540066a0 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.735+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8160047540 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.735+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f8154054950 con 0x7f8160069dd0 2024-04-25T11:01:04.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.739+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f8154035520 con 0x7f8160069dd0 2024-04-25T11:01:04.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.877+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 --> v1:172.21.15.155:6816/43298 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8160121870 con 0x7f8144041b80 2024-04-25T11:01:04.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.878+0000 7f815d7fa700 1 -- 172.21.15.155:0/470834721 <== mgr.4100 v1:172.21.15.155:6816/43298 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32749 (unknown 2965378022 0 3479332488) 0x7f8160121870 con 0x7f8144041b80 2024-04-25T11:01:04.879 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 >> v1:172.21.15.155:6816/43298 conn(0x7f8144041b80 legacy=0x7f8144044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 msgr2=0x7f81601242a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 0x7f81601242a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f815400c4c0 tx=0x7f8154007e00).stop 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 shutdown_connections 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81601289c0 0x7f8160124c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8160069dd0 0x7f81601242a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 --2- 172.21.15.155:0/470834721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81601234e0 0x7f8160124790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:04.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 >> 172.21.15.155:0/470834721 conn(0x7f81600956f0 msgr2=0x7f8160121b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:04.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 shutdown_connections 2024-04-25T11:01:04.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:04.880+0000 7f8166fd7700 1 -- 172.21.15.155:0/470834721 wait complete. 2024-04-25T11:01:04.882 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:01:04.892 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T11:01:03.391982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188796,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298564,"statfs":{"total":386547056640,"available":385329729536,"internally_reserved":0,"allocated":421888,"data_stored":106876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.363290+0000","last_change":"2024-04-25T11:01:01.361046+0000","last_active":"2024-04-25T11:01:01.363290+0000","last_peered":"2024-04-25T11:01:01.363290+0000","last_clean":"2024-04-25T11:01:01.363290+0000","last_became_active":"2024-04-25T11:01:01.360746+0000","last_became_peered":"2024-04-25T11:01:01.360746+0000","last_unstale":"2024-04-25T11:01:01.363290+0000","last_undegraded":"2024-04-25T11:01:01.363290+0000","last_fullsized":"2024-04-25T11:01:01.363290+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.362815+0000","last_change":"2024-04-25T11:01:01.361196+0000","last_active":"2024-04-25T11:01:01.362815+0000","last_peered":"2024-04-25T11:01:01.362815+0000","last_clean":"2024-04-25T11:01:01.362815+0000","last_became_active":"2024-04-25T11:01:01.360891+0000","last_became_peered":"2024-04-25T11:01:01.360891+0000","last_unstale":"2024-04-25T11:01:01.362815+0000","last_undegraded":"2024-04-25T11:01:01.362815+0000","last_fullsized":"2024-04-25T11:01:01.362815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.363015+0000","last_change":"2024-04-25T11:01:01.361055+0000","last_active":"2024-04-25T11:01:01.363015+0000","last_peered":"2024-04-25T11:01:01.363015+0000","last_clean":"2024-04-25T11:01:01.363015+0000","last_became_active":"2024-04-25T11:01:01.360774+0000","last_became_peered":"2024-04-25T11:01:01.360774+0000","last_unstale":"2024-04-25T11:01:01.363015+0000","last_undegraded":"2024-04-25T11:01:01.363015+0000","last_fullsized":"2024-04-25T11:01:01.363015+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.436878+0000","last_change":"2024-04-25T11:01:00.434886+0000","last_active":"2024-04-25T11:01:00.436878+0000","last_peered":"2024-04-25T11:01:00.436878+0000","last_clean":"2024-04-25T11:01:00.436878+0000","last_became_active":"2024-04-25T11:01:00.434536+0000","last_became_peered":"2024-04-25T11:01:00.434536+0000","last_unstale":"2024-04-25T11:01:00.436878+0000","last_undegraded":"2024-04-25T11:01:00.436878+0000","last_fullsized":"2024-04-25T11:01:00.436878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.429443+0000","last_change":"2024-04-25T11:00:59.424526+0000","last_active":"2024-04-25T11:01:00.429443+0000","last_peered":"2024-04-25T11:01:00.429443+0000","last_clean":"2024-04-25T11:01:00.429443+0000","last_became_active":"2024-04-25T11:00:59.424232+0000","last_became_peered":"2024-04-25T11:00:59.424232+0000","last_unstale":"2024-04-25T11:01:00.429443+0000","last_undegraded":"2024-04-25T11:01:00.429443+0000","last_fullsized":"2024-04-25T11:01:00.429443+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.436424+0000","last_change":"2024-04-25T11:01:00.431767+0000","last_active":"2024-04-25T11:01:00.436424+0000","last_peered":"2024-04-25T11:01:00.436424+0000","last_clean":"2024-04-25T11:01:00.436424+0000","last_became_active":"2024-04-25T11:01:00.431464+0000","last_became_peered":"2024-04-25T11:01:00.431464+0000","last_unstale":"2024-04-25T11:01:00.436424+0000","last_undegraded":"2024-04-25T11:01:00.436424+0000","last_fullsized":"2024-04-25T11:01:00.436424+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.429186+0000","last_change":"2024-04-25T11:01:00.429186+0000","last_active":"2024-04-25T11:01:00.429186+0000","last_peered":"2024-04-25T11:01:00.429186+0000","last_clean":"2024-04-25T11:01:00.429186+0000","last_became_active":"2024-04-25T11:01:00.428901+0000","last_became_peered":"2024-04-25T11:01:00.428901+0000","last_unstale":"2024-04-25T11:01:00.429186+0000","last_undegraded":"2024-04-25T11:01:00.429186+0000","last_fullsized":"2024-04-25T11:01:00.429186+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.436870+0000","last_change":"2024-04-25T11:01:00.434904+0000","last_active":"2024-04-25T11:01:00.436870+0000","last_peered":"2024-04-25T11:01:00.436870+0000","last_clean":"2024-04-25T11:01:00.436870+0000","last_became_active":"2024-04-25T11:01:00.434505+0000","last_became_peered":"2024-04-25T11:01:00.434505+0000","last_unstale":"2024-04-25T11:01:00.436870+0000","last_undegraded":"2024-04-25T11:01:00.436870+0000","last_fullsized":"2024-04-25T11:01:00.436870+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.364607+0000","last_change":"2024-04-25T11:01:01.363969+0000","last_active":"2024-04-25T11:01:01.364607+0000","last_peered":"2024-04-25T11:01:01.364607+0000","last_clean":"2024-04-25T11:01:01.364607+0000","last_became_active":"2024-04-25T11:01:01.363581+0000","last_became_peered":"2024-04-25T11:01:01.363581+0000","last_unstale":"2024-04-25T11:01:01.364607+0000","last_undegraded":"2024-04-25T11:01:01.364607+0000","last_fullsized":"2024-04-25T11:01:01.364607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]}]},{"osd":1,"up_from":10,"seq":42949672962,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297188,"kb_used_data":92,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074652,"statfs":{"total":96636764160,"available":96332443648,"internally_reserved":0,"allocated":94208,"data_stored":22244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:01:04.893 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:01:05.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.006+0000 7f5718f12700 1 Processor -- start 2024-04-25T11:01:05.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.006+0000 7f5718f12700 1 -- start start 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5718f12700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57140626c0 0x7f5714125460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5718f12700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f571412a6c0 0x7f571412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5718f12700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57140676e0 con 0x7f5714067bc0 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5718f12700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5714067870 con 0x7f57140626c0 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5718f12700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5714126060 con 0x7f571412a6c0 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571259c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57140626c0 0x7f5714125460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5712d9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f571412a6c0 0x7f571412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5711d9b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5714067bc0 0x7f5714125950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44734/0 (socket says 172.21.15.155:44734) 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5711d9b700 1 -- 172.21.15.155:0/4149637767 learned_addr learned my addr 172.21.15.155:0/4149637767 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571259c700 1 -- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f571412a6c0 msgr2=0x7f571412cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571259c700 1 --2- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f571412a6c0 0x7f571412cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571259c700 1 -- 172.21.15.155:0/4149637767 >> v1:172.21.15.155:6789/0 conn(0x7f5714067bc0 legacy=0x7f5714125950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571159a700 1 -- 172.21.15.155:0/4149637767 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122766054 0 0) 0x7f57140676e0 con 0x7f5714067bc0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571259c700 1 -- 172.21.15.155:0/4149637767 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57141261d0 con 0x7f57140626c0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f5712d9d700 1 --2- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f571412a6c0 0x7f571412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571259c700 1 --2- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57140626c0 0x7f5714125460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5708009760 tx=0x7f5708009060).ready entity=mon.0 client_cookie=7f1d11e71e926e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571159a700 1 -- 172.21.15.155:0/4149637767 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570800f510 con 0x7f57140626c0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.007+0000 7f571159a700 1 -- 172.21.15.155:0/4149637767 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570800f670 con 0x7f57140626c0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f571159a700 1 -- 172.21.15.155:0/4149637767 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570800f9f0 con 0x7f57140626c0 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 -- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57140626c0 msgr2=0x7f5714125460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 --2- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57140626c0 0x7f5714125460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5708009760 tx=0x7f5708009060).stop 2024-04-25T11:01:05.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 -- 172.21.15.155:0/4149637767 shutdown_connections 2024-04-25T11:01:05.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 --2- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f571412a6c0 0x7f571412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 --2- 172.21.15.155:0/4149637767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57140626c0 0x7f5714125460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 -- 172.21.15.155:0/4149637767 >> 172.21.15.155:0/4149637767 conn(0x7f57140956f0 msgr2=0x7f5714095b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:05.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 -- 172.21.15.155:0/4149637767 shutdown_connections 2024-04-25T11:01:05.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.008+0000 7f5718f12700 1 -- 172.21.15.155:0/4149637767 wait complete. 2024-04-25T11:01:05.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.009+0000 7f5718f12700 1 Processor -- start 2024-04-25T11:01:05.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.009+0000 7f5718f12700 1 -- start start 2024-04-25T11:01:05.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f57140626c0 0x7f5714134a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 0x7f5714134f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 0x7f57141353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5714121f50 con 0x7f5714067bc0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5714121320 con 0x7f571412a6c0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5714126f40 con 0x7f57140626c0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 0x7f5714134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5712d9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 0x7f57141353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 0x7f5714134f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34384/0 (socket says 172.21.15.155:34384) 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5712d9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 0x7f57141353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49428/0 (socket says 172.21.15.155:49428) 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 -- 172.21.15.155:0/3993137081 learned_addr learned my addr 172.21.15.155:0/3993137081 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f571259c700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f57140626c0 0x7f5714134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 -- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f57140626c0 msgr2=0x7f5714134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f57140626c0 0x7f5714134a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 -- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 msgr2=0x7f57141353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 0x7f57141353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 -- 172.21.15.155:0/3993137081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c009a20 con 0x7f5714067bc0 2024-04-25T11:01:05.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5712d9d700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 0x7f57141353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f571259c700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f57140626c0 0x7f5714134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5711d9b700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 0x7f5714134f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f570400c4c0 tx=0x7f5704007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040072e0 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704007460 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040059f0 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57080092a0 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.010+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714126b70 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.011+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f5704013450 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.011+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5714047540 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.012+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f57040548b0 con 0x7f5714067bc0 2024-04-25T11:01:05.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.015+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f5704035420 con 0x7f5714067bc0 2024-04-25T11:01:05.152 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.152+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 --> v1:172.21.15.155:6816/43298 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f571412d0a0 con 0x7f56f4041b80 2024-04-25T11:01:05.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.153+0000 7f5702ffd700 1 -- 172.21.15.155:0/3993137081 <== mgr.4100 v1:172.21.15.155:6816/43298 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32749 (unknown 2965378022 0 3479332488) 0x7f571412d0a0 con 0x7f56f4041b80 2024-04-25T11:01:05.154 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:05.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 >> v1:172.21.15.155:6816/43298 conn(0x7f56f4041b80 legacy=0x7f56f4044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 msgr2=0x7f5714134f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 0x7f5714134f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f570400c4c0 tx=0x7f5704007c10).stop 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 shutdown_connections 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f571412a6c0 0x7f57141353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f57140626c0 0x7f5714134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 --2- 172.21.15.155:0/3993137081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5714067bc0 0x7f5714134f00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 >> 172.21.15.155:0/3993137081 conn(0x7f57140956f0 msgr2=0x7f5714065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 shutdown_connections 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.155+0000 7f5718f12700 1 -- 172.21.15.155:0/3993137081 wait complete. 2024-04-25T11:01:05.156 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:01:05.167 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T11:01:03.391982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188796,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298564,"statfs":{"total":386547056640,"available":385329729536,"internally_reserved":0,"allocated":421888,"data_stored":106876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.363290+0000","last_change":"2024-04-25T11:01:01.361046+0000","last_active":"2024-04-25T11:01:01.363290+0000","last_peered":"2024-04-25T11:01:01.363290+0000","last_clean":"2024-04-25T11:01:01.363290+0000","last_became_active":"2024-04-25T11:01:01.360746+0000","last_became_peered":"2024-04-25T11:01:01.360746+0000","last_unstale":"2024-04-25T11:01:01.363290+0000","last_undegraded":"2024-04-25T11:01:01.363290+0000","last_fullsized":"2024-04-25T11:01:01.363290+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.362815+0000","last_change":"2024-04-25T11:01:01.361196+0000","last_active":"2024-04-25T11:01:01.362815+0000","last_peered":"2024-04-25T11:01:01.362815+0000","last_clean":"2024-04-25T11:01:01.362815+0000","last_became_active":"2024-04-25T11:01:01.360891+0000","last_became_peered":"2024-04-25T11:01:01.360891+0000","last_unstale":"2024-04-25T11:01:01.362815+0000","last_undegraded":"2024-04-25T11:01:01.362815+0000","last_fullsized":"2024-04-25T11:01:01.362815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.363015+0000","last_change":"2024-04-25T11:01:01.361055+0000","last_active":"2024-04-25T11:01:01.363015+0000","last_peered":"2024-04-25T11:01:01.363015+0000","last_clean":"2024-04-25T11:01:01.363015+0000","last_became_active":"2024-04-25T11:01:01.360774+0000","last_became_peered":"2024-04-25T11:01:01.360774+0000","last_unstale":"2024-04-25T11:01:01.363015+0000","last_undegraded":"2024-04-25T11:01:01.363015+0000","last_fullsized":"2024-04-25T11:01:01.363015+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.436878+0000","last_change":"2024-04-25T11:01:00.434886+0000","last_active":"2024-04-25T11:01:00.436878+0000","last_peered":"2024-04-25T11:01:00.436878+0000","last_clean":"2024-04-25T11:01:00.436878+0000","last_became_active":"2024-04-25T11:01:00.434536+0000","last_became_peered":"2024-04-25T11:01:00.434536+0000","last_unstale":"2024-04-25T11:01:00.436878+0000","last_undegraded":"2024-04-25T11:01:00.436878+0000","last_fullsized":"2024-04-25T11:01:00.436878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.429443+0000","last_change":"2024-04-25T11:00:59.424526+0000","last_active":"2024-04-25T11:01:00.429443+0000","last_peered":"2024-04-25T11:01:00.429443+0000","last_clean":"2024-04-25T11:01:00.429443+0000","last_became_active":"2024-04-25T11:00:59.424232+0000","last_became_peered":"2024-04-25T11:00:59.424232+0000","last_unstale":"2024-04-25T11:01:00.429443+0000","last_undegraded":"2024-04-25T11:01:00.429443+0000","last_fullsized":"2024-04-25T11:01:00.429443+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.436424+0000","last_change":"2024-04-25T11:01:00.431767+0000","last_active":"2024-04-25T11:01:00.436424+0000","last_peered":"2024-04-25T11:01:00.436424+0000","last_clean":"2024-04-25T11:01:00.436424+0000","last_became_active":"2024-04-25T11:01:00.431464+0000","last_became_peered":"2024-04-25T11:01:00.431464+0000","last_unstale":"2024-04-25T11:01:00.436424+0000","last_undegraded":"2024-04-25T11:01:00.436424+0000","last_fullsized":"2024-04-25T11:01:00.436424+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.429186+0000","last_change":"2024-04-25T11:01:00.429186+0000","last_active":"2024-04-25T11:01:00.429186+0000","last_peered":"2024-04-25T11:01:00.429186+0000","last_clean":"2024-04-25T11:01:00.429186+0000","last_became_active":"2024-04-25T11:01:00.428901+0000","last_became_peered":"2024-04-25T11:01:00.428901+0000","last_unstale":"2024-04-25T11:01:00.429186+0000","last_undegraded":"2024-04-25T11:01:00.429186+0000","last_fullsized":"2024-04-25T11:01:00.429186+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:00.436870+0000","last_change":"2024-04-25T11:01:00.434904+0000","last_active":"2024-04-25T11:01:00.436870+0000","last_peered":"2024-04-25T11:01:00.436870+0000","last_clean":"2024-04-25T11:01:00.436870+0000","last_became_active":"2024-04-25T11:01:00.434505+0000","last_became_peered":"2024-04-25T11:01:00.434505+0000","last_unstale":"2024-04-25T11:01:00.436870+0000","last_undegraded":"2024-04-25T11:01:00.436870+0000","last_fullsized":"2024-04-25T11:01:00.436870+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-25T11:01:01.364607+0000","last_change":"2024-04-25T11:01:01.363969+0000","last_active":"2024-04-25T11:01:01.364607+0000","last_peered":"2024-04-25T11:01:01.364607+0000","last_clean":"2024-04-25T11:01:01.364607+0000","last_became_active":"2024-04-25T11:01:01.363581+0000","last_became_peered":"2024-04-25T11:01:01.363581+0000","last_unstale":"2024-04-25T11:01:01.364607+0000","last_undegraded":"2024-04-25T11:01:01.364607+0000","last_fullsized":"2024-04-25T11:01:01.364607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]}]},{"osd":1,"up_from":10,"seq":42949672962,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297188,"kb_used_data":92,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074652,"statfs":{"total":96636764160,"available":96332443648,"internally_reserved":0,"allocated":94208,"data_stored":22244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:01:05.168 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:01:05.168 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:01:05.168 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:01:05.168 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:01:05.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.281+0000 7f12cc6e1700 1 Processor -- start 2024-04-25T11:01:05.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.281+0000 7f12cc6e1700 1 -- start start 2024-04-25T11:01:05.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12cc6e1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 0x7f12c41221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12cc6e1700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12c4065300 0x7f12c41226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12cc6e1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c4064ba0 con 0x7f12c4122bd0 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12cc6e1700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c4066f40 con 0x7f12c4065300 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12cc6e1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40678d0 con 0x7f12c4093130 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12ca47d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 0x7f12c41221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c9c7c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12c4065300 0x7f12c41226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12ca47d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 0x7f12c41221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49442/0 (socket says 172.21.15.155:49442) 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c9c7c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12c4065300 0x7f12c41226e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34398/0 (socket says 172.21.15.155:34398) 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12ca47d700 1 -- 172.21.15.155:0/3077742899 learned_addr learned my addr 172.21.15.155:0/3077742899 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12cac7e700 1 --1- 172.21.15.155:0/3077742899 >> v1:172.21.15.155:6789/0 conn(0x7f12c4122bd0 0x7f12c4125540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44746/0 (socket says 172.21.15.155:44746) 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c9c7c700 1 -- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 msgr2=0x7f12c41221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:05.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c9c7c700 1 --2- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 0x7f12c41221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c9c7c700 1 -- 172.21.15.155:0/3077742899 >> v1:172.21.15.155:6789/0 conn(0x7f12c4122bd0 legacy=0x7f12c4125540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c9c7c700 1 -- 172.21.15.155:0/3077742899 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4125c50 con 0x7f12c4065300 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12c947b700 1 -- 172.21.15.155:0/3077742899 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378114447 0 0) 0x7f12c4064ba0 con 0x7f12c4122bd0 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.282+0000 7f12ca47d700 1 --2- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 0x7f12c41221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12c9c7c700 1 --2- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4065300 0x7f12c41226e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f12c0009760 tx=0x7f12c0009060).ready entity=mon.0 client_cookie=bab84c00482d065d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12c947b700 1 -- 172.21.15.155:0/3077742899 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000f510 con 0x7f12c4065300 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12c947b700 1 -- 172.21.15.155:0/3077742899 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000f670 con 0x7f12c4065300 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12c947b700 1 -- 172.21.15.155:0/3077742899 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000f9f0 con 0x7f12c4065300 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4065300 msgr2=0x7f12c41226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4065300 0x7f12c41226e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f12c0009760 tx=0x7f12c0009060).stop 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3077742899 shutdown_connections 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4093130 0x7f12c41221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3077742899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4065300 0x7f12c41226e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3077742899 >> 172.21.15.155:0/3077742899 conn(0x7f12c4096a60 msgr2=0x7f12c4091a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:05.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3077742899 shutdown_connections 2024-04-25T11:01:05.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.283+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3077742899 wait complete. 2024-04-25T11:01:05.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.284+0000 7f12cc6e1700 1 Processor -- start 2024-04-25T11:01:05.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.284+0000 7f12cc6e1700 1 -- start start 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.284+0000 7f12cc6e1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4065300 0x7f12c4116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f12c4093130 0x7f12c4116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 0x7f12c4116bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4117f00 con 0x7f12c4122bd0 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4121d40 con 0x7f12c4065300 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4119730 con 0x7f12c4093130 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cac7e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 0x7f12c4116bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12c9c7c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f12c4093130 0x7f12c4116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4065300 0x7f12c4116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12c9c7c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f12c4093130 0x7f12c4116700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:47418/0 (socket says 172.21.15.155:47418) 2024-04-25T11:01:05.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cac7e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 0x7f12c4116bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34402/0 (socket says 172.21.15.155:34402) 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12c9c7c700 1 -- 172.21.15.155:0/3742388520 learned_addr learned my addr 172.21.15.155:0/3742388520 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 -- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f12c4093130 msgr2=0x7f12c4116700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f12c4093130 0x7f12c4116700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 -- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 msgr2=0x7f12c4116bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 0x7f12c4116bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 -- 172.21.15.155:0/3742388520 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4009370 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cac7e700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 0x7f12c4116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12ca47d700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4065300 0x7f12c4116210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f12b400ea00 tx=0x7f12b40090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4009650 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b400c6a0 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b400ca20 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c00092a0 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.285+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c4118f50 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.286+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f12b401b400 con 0x7f12c4065300 2024-04-25T11:01:05.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.286+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c4047540 con 0x7f12c4065300 2024-04-25T11:01:05.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.286+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f12b40556a0 con 0x7f12c4065300 2024-04-25T11:01:05.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.290+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f12b4036040 con 0x7f12c4065300 2024-04-25T11:01:05.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.455+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12c4126080 con 0x7f12c4065300 2024-04-25T11:01:05.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.456+0000 7f12baffd700 1 -- 172.21.15.155:0/3742388520 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f12b4039830 con 0x7f12c4065300 2024-04-25T11:01:05.457 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 >> v1:172.21.15.155:6816/43298 conn(0x7f12a8041b80 legacy=0x7f12a8044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4065300 msgr2=0x7f12c4116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4065300 0x7f12c4116210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f12b400ea00 tx=0x7f12b40090f0).stop 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 shutdown_connections 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4122bd0 0x7f12c4116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12c4065300 0x7f12c4116210 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 --2- 172.21.15.155:0/3742388520 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f12c4093130 0x7f12c4116700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 >> 172.21.15.155:0/3742388520 conn(0x7f12c4096a60 msgr2=0x7f12c4123eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 shutdown_connections 2024-04-25T11:01:05.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.457+0000 7f12cc6e1700 1 -- 172.21.15.155:0/3742388520 wait complete. 2024-04-25T11:01:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:01:05.470 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:01:05.470 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T11:01:05.484 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:01:05.484 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:01:05.484 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client pacific' 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 Processor -- start 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 -- start start 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44812aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa44812af90 0x7fa44812d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa448068130 con 0x7fa448066240 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa448067110 con 0x7fa44812af90 2024-04-25T11:01:05.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44fa6b700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa448068590 con 0x7fa448066d30 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44d006700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44812aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44d807700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa448066240 0x7fa448066620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44752/0 (socket says 172.21.15.155:44752) 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44d807700 1 -- 172.21.15.155:0/324218981 learned_addr learned my addr 172.21.15.155:0/324218981 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.596+0000 7fa44e008700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa44812af90 0x7fa44812d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44c805700 1 -- 172.21.15.155:0/324218981 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724275490 0 0) 0x7fa448068130 con 0x7fa448066240 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44d006700 1 -- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa44812af90 msgr2=0x7fa44812d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44d006700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa44812af90 0x7fa44812d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44d006700 1 -- 172.21.15.155:0/324218981 >> v1:172.21.15.155:6789/0 conn(0x7fa448066240 legacy=0x7fa448066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44d006700 1 -- 172.21.15.155:0/324218981 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448068b70 con 0x7fa448066d30 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44e008700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa44812af90 0x7fa44812d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44d006700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44812aaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa4440097c0 tx=0x7fa444009060).ready entity=mon.2 client_cookie=c75f1f1b4e6d2bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:05.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44c805700 1 -- 172.21.15.155:0/324218981 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440090d0 con 0x7fa448066d30 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44c805700 1 -- 172.21.15.155:0/324218981 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44400f510 con 0x7fa448066d30 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44c805700 1 -- 172.21.15.155:0/324218981 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400f890 con 0x7fa448066d30 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44fa6b700 1 -- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 msgr2=0x7fa44812aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44812aaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa4440097c0 tx=0x7fa444009060).stop 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44fa6b700 1 -- 172.21.15.155:0/324218981 shutdown_connections 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa44812af90 0x7fa44812d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/324218981 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44812aaa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.597+0000 7fa44fa6b700 1 -- 172.21.15.155:0/324218981 >> 172.21.15.155:0/324218981 conn(0x7fa448091ca0 msgr2=0x7fa448094670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.598+0000 7fa44fa6b700 1 -- 172.21.15.155:0/324218981 shutdown_connections 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.598+0000 7fa44fa6b700 1 -- 172.21.15.155:0/324218981 wait complete. 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.598+0000 7fa44fa6b700 1 Processor -- start 2024-04-25T11:01:05.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.598+0000 7fa44fa6b700 1 -- start start 2024-04-25T11:01:05.603 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 0x7fa44811a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44811a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44812af90 0x7fa44811ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44811d990 con 0x7fa44812af90 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44811d010 con 0x7fa448066240 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa44811dbc0 con 0x7fa448066d30 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 0x7fa44811a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 0x7fa44811a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49476/0 (socket says 172.21.15.155:49476) 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 -- 172.21.15.155:0/94912637 learned_addr learned my addr 172.21.15.155:0/94912637 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d006700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44811a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44e008700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44812af90 0x7fa44811ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 -- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 msgr2=0x7fa44811a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44811a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 -- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44812af90 msgr2=0x7fa44811ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44812af90 0x7fa44811ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 -- 172.21.15.155:0/94912637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440009a20 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44d807700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 0x7fa44811a360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa43800c4e0 tx=0x7fa438007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800ce30 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4380072e0 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438005670 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.599+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4440092a0 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.600+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44811bed0 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.601+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7fa438005a20 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.601+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44811d280 con 0x7fa448066240 2024-04-25T11:01:05.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.601+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fa438054720 con 0x7fa448066240 2024-04-25T11:01:05.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.604+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7fa438035120 con 0x7fa448066240 2024-04-25T11:01:05.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:05.752+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "pacific"} v 0) v1 -- 0x7fa448090f00 con 0x7fa448066240 2024-04-25T11:01:06.480 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.480+0000 7fa43e777700 1 -- 172.21.15.155:0/94912637 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "pacific"}]=0 set require_min_compat_client to pacific v14) v1 ==== 143+0+0 (secure 0 0 0) 0x7fa438018020 con 0x7fa448066240 2024-04-25T11:01:06.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 >> v1:172.21.15.155:6816/43298 conn(0x7fa42c041b80 legacy=0x7fa42c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 msgr2=0x7fa44811a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 0x7fa44811a360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa43800c4e0 tx=0x7fa438007a80).stop 2024-04-25T11:01:06.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 shutdown_connections 2024-04-25T11:01:06.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44812af90 0x7fa44811ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa448066240 0x7fa44811a360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 --2- 172.21.15.155:0/94912637 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa448066d30 0x7fa44811a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 >> 172.21.15.155:0/94912637 conn(0x7fa448091ca0 msgr2=0x7fa448089bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:06.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 shutdown_connections 2024-04-25T11:01:06.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.482+0000 7fa44fa6b700 1 -- 172.21.15.155:0/94912637 wait complete. 2024-04-25T11:01:06.484 INFO:teuthology.orchestra.run.smithi155.stderr:set require_min_compat_client to pacific 2024-04-25T11:01:06.496 INFO:teuthology.run_tasks:Running task print... 2024-04-25T11:01:06.505 INFO:teuthology.task.print:**** done ceph 2024-04-25T11:01:06.505 INFO:teuthology.run_tasks:Running task print... 2024-04-25T11:01:06.574 INFO:teuthology.task.print:*** upgrading, no cephfs present 2024-04-25T11:01:06.574 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T11:01:06.617 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:01:06.617 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:01:06.617 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 Processor -- start 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 -- start start 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60122390 0x7f5e60122770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60069560 0x7f5e60122c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e60069cd0 con 0x7f5e600672d0 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6005e430 con 0x7f5e60122390 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e65e29700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6005e5a0 con 0x7f5e60069560 2024-04-25T11:01:06.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e5f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60122390 0x7f5e60122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:06.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e5effd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60069560 0x7f5e60122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:06.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e5ffff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5e600672d0 0x7f5e60067750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44762/0 (socket says 172.21.15.155:44762) 2024-04-25T11:01:06.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.742+0000 7f5e5ffff700 1 -- 172.21.15.155:0/2923808898 learned_addr learned my addr 172.21.15.155:0/2923808898 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:01:06.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60069560 msgr2=0x7f5e60122c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:06.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5f7fe700 1 --2- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60069560 0x7f5e60122c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/2923808898 >> v1:172.21.15.155:6789/0 conn(0x7f5e600672d0 legacy=0x7f5e60067750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5e7fc700 1 -- 172.21.15.155:0/2923808898 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591320359 0 0) 0x7f5e60069cd0 con 0x7f5e600672d0 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/2923808898 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60068670 con 0x7f5e60122390 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5effd700 1 --2- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60069560 0x7f5e60122c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5f7fe700 1 --2- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60122390 0x7f5e60122770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f5e500097c0 tx=0x7f5e50009060).ready entity=mon.1 client_cookie=2dceae7901070c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5e7fc700 1 -- 172.21.15.155:0/2923808898 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500090d0 con 0x7f5e60122390 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5e7fc700 1 -- 172.21.15.155:0/2923808898 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5000f510 con 0x7f5e60122390 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e5e7fc700 1 -- 172.21.15.155:0/2923808898 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000f890 con 0x7f5e60122390 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.743+0000 7f5e65e29700 1 -- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60122390 msgr2=0x7f5e60122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 --2- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60122390 0x7f5e60122770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f5e500097c0 tx=0x7f5e50009060).stop 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 -- 172.21.15.155:0/2923808898 shutdown_connections 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 --2- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60122390 0x7f5e60122770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 --2- 172.21.15.155:0/2923808898 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60069560 0x7f5e60122c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 -- 172.21.15.155:0/2923808898 >> 172.21.15.155:0/2923808898 conn(0x7f5e60091e20 msgr2=0x7f5e60094690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:06.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 -- 172.21.15.155:0/2923808898 shutdown_connections 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 -- 172.21.15.155:0/2923808898 wait complete. 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.744+0000 7f5e65e29700 1 Processor -- start 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 -- start start 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 0x7f5e60085b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60122390 0x7f5e60086080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60086570 0x7f5e60134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6008c310 con 0x7f5e60069560 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6008c980 con 0x7f5e60086570 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e65e29700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6008c6c0 con 0x7f5e60122390 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 0x7f5e60085b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5ffff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60086570 0x7f5e60134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 0x7f5e60085b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:34412/0 (socket says 172.21.15.155:34412) 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/307474730 learned_addr learned my addr 172.21.15.155:0/307474730 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5effd700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60122390 0x7f5e60086080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-25T11:01:06.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60122390 msgr2=0x7f5e60086080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60122390 0x7f5e60086080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60086570 msgr2=0x7f5e60134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60086570 0x7f5e60134890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5f7fe700 1 -- 172.21.15.155:0/307474730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540092a0 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.745+0000 7f5e5ffff700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60086570 0x7f5e60134890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.746+0000 7f5e5f7fe700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 0x7f5e60085b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f5e50009e60 tx=0x7f5e50017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.746+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000f890 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.746+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5000fc50 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.746+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e500092a0 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.746+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5001c420 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.746+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e60086a30 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.747+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) v1 ==== 48491+0+0 (secure 0 0 0) 0x7f5e5001c810 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.747+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e60065b10 con 0x7f5e60069560 2024-04-25T11:01:06.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.747+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f5e500568b0 con 0x7f5e60069560 2024-04-25T11:01:06.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.750+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+131111 (secure 0 0 0) 0x7f5e50026d70 con 0x7f5e60069560 2024-04-25T11:01:06.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.913+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5e60066d20 con 0x7f5e60069560 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.914+0000 7f5e47fff700 1 -- 172.21.15.155:0/307474730 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7f5e50020030 con 0x7f5e60069560 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stdout:e1 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stdout:legacy client fscid: -1 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:06.915 INFO:teuthology.orchestra.run.smithi155.stdout:No filesystems configured 2024-04-25T11:01:06.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 >> v1:172.21.15.155:6816/43298 conn(0x7f5e4c041b80 legacy=0x7f5e4c044040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 msgr2=0x7f5e60085b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 0x7f5e60085b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f5e50009e60 tx=0x7f5e50017000).stop 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 shutdown_connections 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e60086570 0x7f5e60134890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e60069560 0x7f5e60085b90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 --2- 172.21.15.155:0/307474730 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e60122390 0x7f5e60086080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 >> 172.21.15.155:0/307474730 conn(0x7f5e600672d0 msgr2=0x7f5e6011b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 shutdown_connections 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:01:06.915+0000 7f5e65e29700 1 -- 172.21.15.155:0/307474730 wait complete. 2024-04-25T11:01:06.917 INFO:teuthology.orchestra.run.smithi155.stderr:dumped fsmap epoch 1 2024-04-25T11:01:06.927 INFO:teuthology.run_tasks:Running task install.upgrade... 2024-04-25T11:01:06.937 INFO:teuthology.task.install:project ceph config {'mon.a': {'branch': 'reef'}} overrides {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T11:01:06.937 INFO:teuthology.task.install:config contains sha1|tag|branch, removing those keys from override 2024-04-25T11:01:06.937 INFO:teuthology.task.install:remote ubuntu@smithi155.front.sepia.ceph.com config {'branch': 'reef', 'flavor': 'default'} 2024-04-25T11:01:06.938 INFO:teuthology.task.install:extra packages: [] 2024-04-25T11:01:06.944 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:01:06.944 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2024-04-25T11:01:06.944 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:01:06.966 INFO:teuthology.orchestra.run.smithi155.stdout:16.2.15-0.el8 2024-04-25T11:01:06.966 INFO:teuthology.packaging:The installed version of ceph-common is 16.2.15-0.el8 2024-04-25T11:01:06.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-04-25T11:01:07.191 INFO:teuthology.task.install:Ceph rpm upgrade from 16.2.15-0.el8 to 18.2.3-0 2024-04-25T11:01:07.191 INFO:teuthology.task.install.rpm:Host smithi155 is: centos 8 x86_64 2024-04-25T11:01:07.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-04-25T11:01:07.519 INFO:teuthology.task.install.rpm:Repo base URL: https://chacra.ceph.com/r/ceph/reef/76424b2fe1bb19c32c52140f39764599abf5e035/centos/8/flavors/default/ 2024-04-25T11:01:07.520 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/yum.repos.d/ceph.repo 2024-04-25T11:01:07.582 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/76424b2fe1bb19c32c52140f39764599abf5e035/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/76424b2fe1bb19c32c52140f39764599abf5e035/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/76424b2fe1bb19c32c52140f39764599abf5e035/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:01:07.582 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T11:01:07.582 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:01:07.616 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:01:07.685 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:01:07.753 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:01:07.811 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-04-25T11:01:07.822 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-04-25T11:01:08.081 INFO:teuthology.orchestra.run.smithi155.stdout:71 files removed 2024-04-25T11:01:08.110 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2024-04-25T11:01:08.672 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 545 kB/s | 80 kB 00:00 2024-04-25T11:01:08.847 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 82 kB/s | 11 kB 00:00 2024-04-25T11:01:08.990 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 16 kB/s | 1.8 kB 00:00 2024-04-25T11:01:09.690 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - AppStream 42 MB/s | 28 MB 00:00 2024-04-25T11:01:15.241 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - BaseOS 20 MB/s | 10 MB 00:00 2024-04-25T11:01:17.133 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - Extras 63 kB/s | 18 kB 00:00 2024-04-25T11:01:17.476 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - Extras common packages 33 kB/s | 7.7 kB 00:00 2024-04-25T11:01:19.486 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 8 - PowerTools 2.6 MB/s | 5.0 MB 00:01 2024-04-25T11:01:20.713 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2024-04-25T11:01:24.808 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 16 MB 00:03 2024-04-25T11:01:28.664 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 426 kB/s | 24 kB 00:00 2024-04-25T11:01:32.353 INFO:teuthology.orchestra.run.smithi155.stderr:Package cephadm available, but not installed. 2024-04-25T11:01:32.354 INFO:teuthology.orchestra.run.smithi155.stdout:No match for argument: cephadm 2024-04-25T11:01:32.359 INFO:teuthology.orchestra.run.smithi155.stderr:Package ceph-mgr-dashboard available, but not installed. 2024-04-25T11:01:32.359 INFO:teuthology.orchestra.run.smithi155.stdout:No match for argument: ceph-mgr-dashboard 2024-04-25T11:01:32.360 INFO:teuthology.orchestra.run.smithi155.stderr:Package ceph-mgr-diskprediction-local available, but not installed. 2024-04-25T11:01:32.360 INFO:teuthology.orchestra.run.smithi155.stdout:No match for argument: ceph-mgr-diskprediction-local 2024-04-25T11:01:32.360 INFO:teuthology.orchestra.run.smithi155.stderr:Package ceph-mgr-rook available, but not installed. 2024-04-25T11:01:32.361 INFO:teuthology.orchestra.run.smithi155.stdout:No match for argument: ceph-mgr-rook 2024-04-25T11:01:32.361 INFO:teuthology.orchestra.run.smithi155.stderr:Package ceph-mgr-cephadm available, but not installed. 2024-04-25T11:01:32.361 INFO:teuthology.orchestra.run.smithi155.stdout:No match for argument: ceph-mgr-cephadm 2024-04-25T11:01:32.363 INFO:teuthology.orchestra.run.smithi155.stderr:Package ceph-volume available, but not installed. 2024-04-25T11:01:32.364 INFO:teuthology.orchestra.run.smithi155.stdout:No match for argument: ceph-volume 2024-04-25T11:01:32.468 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.2.3-0.el8 ceph 6.3 k 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.2.3-0.el8 ceph 5.5 M 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.2.3-0.el8 ceph 22 M 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.2.3-0.el8 ceph 859 k 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.2.3-0.el8 ceph 165 k 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.2.3-0.el8 ceph 2.3 M 2024-04-25T11:01:32.473 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.2.3-0.el8 ceph 1.6 M 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.2.3-0.el8 ceph-noarch 245 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.2.3-0.el8 ceph 4.8 M 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.2.3-0.el8 ceph 18 M 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.2.3-0.el8 ceph 13 M 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.2.3-0.el8 ceph 24 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.2.3-0.el8 ceph 47 M 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.2.3-0.el8 ceph 31 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.2.3-0.el8 ceph 781 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.2.3-0.el8 ceph 142 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.2.3-0.el8 ceph 139 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.2.3-0.el8 ceph 3.7 M 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.2.3-0.el8 ceph 474 k 2024-04-25T11:01:32.474 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.2.3-0.el8 ceph 3.6 M 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.2.3-0.el8 ceph 6.6 M 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.2.3-0.el8 ceph 45 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.2.3-0.el8 ceph 129 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.2.3-0.el8 ceph 197 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.2.3-0.el8 ceph 370 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.2.3-0.el8 ceph 364 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.2.3-0.el8 ceph 106 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.2.3-0.el8 ceph 95 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.2.3-0.el8 ceph 3.5 M 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.2.3-0.el8 ceph 181 k 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-25T11:01:32.475 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 30 Packages 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 138 M 2024-04-25T11:01:32.476 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-25T11:01:38.332 INFO:teuthology.orchestra.run.smithi155.stdout:(1/31): ceph-18.2.3-0.el8.x86_64.rpm 122 kB/s | 6.3 kB 00:00 2024-04-25T11:01:38.533 INFO:teuthology.orchestra.run.smithi155.stdout:(2/31): ceph-base-18.2.3-0.el8.x86_64.rpm 22 MB/s | 5.5 MB 00:00 2024-04-25T11:01:38.624 INFO:teuthology.orchestra.run.smithi155.stdout:(3/31): ceph-fuse-18.2.3-0.el8.x86_64.rpm 9.3 MB/s | 859 kB 00:00 2024-04-25T11:01:38.894 INFO:teuthology.orchestra.run.smithi155.stdout:(4/31): ceph-immutable-object-cache-18.2.3-0.el 614 kB/s | 165 kB 00:00 2024-04-25T11:01:39.008 INFO:teuthology.orchestra.run.smithi155.stdout:(5/31): ceph-mds-18.2.3-0.el8.x86_64.rpm 20 MB/s | 2.3 MB 00:00 2024-04-25T11:01:39.050 INFO:teuthology.orchestra.run.smithi155.stdout:(6/31): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2024-04-25T11:01:39.376 INFO:teuthology.orchestra.run.smithi155.stdout:(7/31): ceph-common-18.2.3-0.el8.x86_64.rpm 21 MB/s | 22 MB 00:01 2024-04-25T11:01:39.418 INFO:teuthology.orchestra.run.smithi155.stdout:(8/31): ceph-mgr-18.2.3-0.el8.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-04-25T11:01:39.718 INFO:teuthology.orchestra.run.smithi155.stdout:(9/31): ceph-mon-18.2.3-0.el8.x86_64.rpm 7.2 MB/s | 4.8 MB 00:00 2024-04-25T11:01:39.919 INFO:teuthology.orchestra.run.smithi155.stdout:(10/31): ceph-osd-18.2.3-0.el8.x86_64.rpm 34 MB/s | 18 MB 00:00 2024-04-25T11:01:39.964 INFO:teuthology.orchestra.run.smithi155.stdout:(11/31): ceph-selinux-18.2.3-0.el8.x86_64.rpm 99 kB/s | 24 kB 00:00 2024-04-25T11:01:40.395 INFO:teuthology.orchestra.run.smithi155.stdout:(12/31): libcephfs-devel-18.2.3-0.el8.x86_64.rp 72 kB/s | 31 kB 00:00 2024-04-25T11:01:40.537 INFO:teuthology.orchestra.run.smithi155.stdout:(13/31): ceph-radosgw-18.2.3-0.el8.x86_64.rpm 12 MB/s | 13 MB 00:01 2024-04-25T11:01:40.571 INFO:teuthology.orchestra.run.smithi155.stdout:(14/31): libcephfs2-18.2.3-0.el8.x86_64.rpm 4.4 MB/s | 781 kB 00:00 2024-04-25T11:01:40.595 INFO:teuthology.orchestra.run.smithi155.stdout:(15/31): libcephsqlite-18.2.3-0.el8.x86_64.rpm 2.4 MB/s | 142 kB 00:00 2024-04-25T11:01:40.612 INFO:teuthology.orchestra.run.smithi155.stdout:(16/31): librados-devel-18.2.3-0.el8.x86_64.rpm 3.3 MB/s | 139 kB 00:00 2024-04-25T11:01:40.738 INFO:teuthology.orchestra.run.smithi155.stdout:(17/31): librados2-18.2.3-0.el8.x86_64.rpm 26 MB/s | 3.7 MB 00:00 2024-04-25T11:01:40.763 INFO:teuthology.orchestra.run.smithi155.stdout:(18/31): libradosstriper1-18.2.3-0.el8.x86_64.r 3.1 MB/s | 474 kB 00:00 2024-04-25T11:01:40.838 INFO:teuthology.orchestra.run.smithi155.stdout:(19/31): librbd1-18.2.3-0.el8.x86_64.rpm 36 MB/s | 3.6 MB 00:00 2024-04-25T11:01:40.855 INFO:teuthology.orchestra.run.smithi155.stdout:(20/31): python3-ceph-argparse-18.2.3-0.el8.x86 2.6 MB/s | 45 kB 00:00 2024-04-25T11:01:41.314 INFO:teuthology.orchestra.run.smithi155.stdout:(21/31): ceph-test-18.2.3-0.el8.x86_64.rpm 34 MB/s | 47 MB 00:01 2024-04-25T11:01:41.364 INFO:teuthology.orchestra.run.smithi155.stdout:(22/31): python3-ceph-common-18.2.3-0.el8.x86_6 253 kB/s | 129 kB 00:00 2024-04-25T11:01:41.390 INFO:teuthology.orchestra.run.smithi155.stdout:(23/31): python3-cephfs-18.2.3-0.el8.x86_64.rpm 2.6 MB/s | 197 kB 00:00 2024-04-25T11:01:41.415 INFO:teuthology.orchestra.run.smithi155.stdout:(24/31): python3-rados-18.2.3-0.el8.x86_64.rpm 7.2 MB/s | 370 kB 00:00 2024-04-25T11:01:41.440 INFO:teuthology.orchestra.run.smithi155.stdout:(25/31): python3-rbd-18.2.3-0.el8.x86_64.rpm 7.1 MB/s | 364 kB 00:00 2024-04-25T11:01:41.466 INFO:teuthology.orchestra.run.smithi155.stdout:(26/31): python3-rgw-18.2.3-0.el8.x86_64.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T11:01:41.491 INFO:teuthology.orchestra.run.smithi155.stdout:(27/31): rbd-fuse-18.2.3-0.el8.x86_64.rpm 1.8 MB/s | 95 kB 00:00 2024-04-25T11:01:41.592 INFO:teuthology.orchestra.run.smithi155.stdout:(28/31): librgw2-18.2.3-0.el8.x86_64.rpm 7.9 MB/s | 6.6 MB 00:00 2024-04-25T11:01:41.617 INFO:teuthology.orchestra.run.smithi155.stdout:(29/31): rbd-nbd-18.2.3-0.el8.x86_64.rpm 1.4 MB/s | 181 kB 00:00 2024-04-25T11:01:41.642 INFO:teuthology.orchestra.run.smithi155.stdout:(30/31): ceph-mgr-modules-core-18.2.3-0.el8.noa 4.8 MB/s | 245 kB 00:00 2024-04-25T11:01:41.743 INFO:teuthology.orchestra.run.smithi155.stdout:(31/31): rbd-mirror-18.2.3-0.el8.x86_64.rpm 13 MB/s | 3.5 MB 00:00 2024-04-25T11:01:41.745 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:01:41.746 INFO:teuthology.orchestra.run.smithi155.stdout:Total 15 MB/s | 138 MB 00:09 2024-04-25T11:01:41.746 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-25T11:01:41.807 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-25T11:01:41.807 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-25T11:01:42.469 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-25T11:01:42.472 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-25T11:01:44.575 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-25T11:01:44.755 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2024-04-25T11:01:45.363 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/61 2024-04-25T11:01:45.634 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.2.3-0.el8.x86_64 2/61 2024-04-25T11:01:46.044 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.2.3-0.el8.x86_64 2/61 2024-04-25T11:01:46.258 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.2.3-0.el8.x86_64 3/61 2024-04-25T11:01:46.381 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.2.3-0.el8.x86_64 3/61 2024-04-25T11:01:46.634 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libcephfs2-2:18.2.3-0.el8.x86_64 4/61 2024-04-25T11:01:47.411 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.2.3-0.el8.x86_64 4/61 2024-04-25T11:01:47.667 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librgw2-2:18.2.3-0.el8.x86_64 5/61 2024-04-25T11:01:47.762 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.2.3-0.el8.x86_64 5/61 2024-04-25T11:01:48.038 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : python3-rados-2:18.2.3-0.el8.x86_64 6/61 2024-04-25T11:01:48.242 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libcephsqlite-2:18.2.3-0.el8.x86_64 7/61 2024-04-25T11:01:48.332 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.2.3-0.el8.x86_64 7/61 2024-04-25T11:01:48.550 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libradosstriper1-2:18.2.3-0.el8.x86_64 8/61 2024-04-25T11:01:48.622 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.2.3-0.el8.x86_64 8/61 2024-04-25T11:01:48.861 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : python3-ceph-argparse-2:18.2.3-0.el8.x86_64 9/61 2024-04-25T11:01:49.143 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : python3-cephfs-2:18.2.3-0.el8.x86_64 10/61 2024-04-25T11:01:49.396 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : python3-rbd-2:18.2.3-0.el8.x86_64 11/61 2024-04-25T11:01:49.641 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : python3-rgw-2:18.2.3-0.el8.x86_64 12/61 2024-04-25T11:01:50.135 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados-devel-2:18.2.3-0.el8.x86_64 13/61 2024-04-25T11:01:50.584 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.3-0.el8.noarch 14/61 2024-04-25T11:01:50.811 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : python3-ceph-common-2:18.2.3-0.el8.x86_64 15/61 2024-04-25T11:01:53.103 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.2.3-0.el8.x86_64 16/61 2024-04-25T11:01:53.417 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-common-2:18.2.3-0.el8.x86_64 16/61 2024-04-25T11:01:54.130 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.2.3-0.el8.x86_64 16/61 2024-04-25T11:01:54.465 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-base-2:18.2.3-0.el8.x86_64 17/61 2024-04-25T11:01:54.534 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.2.3-0.el8.x86_64 17/61 2024-04-25T11:01:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-selinux-2:18.2.3-0.el8.x86_64 18/61 2024-04-25T11:02:13.253 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.2.3-0.el8.x86_64 18/61 2024-04-25T11:02:13.506 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-mds-2:18.2.3-0.el8.x86_64 19/61 2024-04-25T11:02:13.654 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.2.3-0.el8.x86_64 19/61 2024-04-25T11:02:13.954 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-mgr-2:18.2.3-0.el8.x86_64 20/61 2024-04-25T11:02:14.497 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.2.3-0.el8.x86_64 20/61 2024-04-25T11:02:14.790 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-mon-2:18.2.3-0.el8.x86_64 21/61 2024-04-25T11:02:16.766 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.2.3-0.el8.x86_64 21/61 2024-04-25T11:02:17.029 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-osd-2:18.2.3-0.el8.x86_64 22/61 2024-04-25T11:02:17.039 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.2.3-0.el8.x86_64 22/61 2024-04-25T11:02:18.598 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-2:18.2.3-0.el8.x86_64 23/61 2024-04-25T11:02:18.863 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-radosgw-2:18.2.3-0.el8.x86_64 24/61 2024-04-25T11:02:18.883 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.2.3-0.el8.x86_64 24/61 2024-04-25T11:02:19.129 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-immutable-object-cache-2:18.2.3-0.el8.x86_64 25/61 2024-04-25T11:02:19.545 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.3-0.el8.x86_64 25/61 2024-04-25T11:02:19.821 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : rbd-mirror-2:18.2.3-0.el8.x86_64 26/61 2024-04-25T11:02:24.957 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.2.3-0.el8.x86_64 26/61 2024-04-25T11:02:25.258 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-test-2:18.2.3-0.el8.x86_64 27/61 2024-04-25T11:02:25.493 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libcephfs-devel-2:18.2.3-0.el8.x86_64 28/61 2024-04-25T11:02:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : rbd-fuse-2:18.2.3-0.el8.x86_64 29/61 2024-04-25T11:02:26.046 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : rbd-nbd-2:18.2.3-0.el8.x86_64 30/61 2024-04-25T11:02:26.330 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : ceph-fuse-2:18.2.3-0.el8.x86_64 31/61 2024-04-25T11:02:26.464 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-2:16.2.15-0.el8.x86_64 32/61 2024-04-25T11:02:26.679 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-test-2:16.2.15-0.el8.x86_64 33/61 2024-04-25T11:02:26.689 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:16.2.15-0.el8.x86_64 34/61 2024-04-25T11:02:26.689 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-radosgw-2:16.2.15-0.el8.x86_64 34/61 2024-04-25T11:02:26.970 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:16.2.15-0.el8.x86_64 34/61 2024-04-25T11:02:27.162 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : rbd-nbd-2:16.2.15-0.el8.x86_64 35/61 2024-04-25T11:02:27.169 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:16.2.15-0.el8.x86_64 36/61 2024-04-25T11:02:27.169 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : rbd-mirror-2:16.2.15-0.el8.x86_64 36/61 2024-04-25T11:02:27.346 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:16.2.15-0.el8.x86_64 36/61 2024-04-25T11:02:27.546 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : rbd-fuse-2:16.2.15-0.el8.x86_64 37/61 2024-04-25T11:02:27.552 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 38/61 2024-04-25T11:02:27.553 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 38/61 2024-04-25T11:02:27.729 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 38/61 2024-04-25T11:02:27.904 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libcephfs-devel-2:16.2.15-0.el8.x86_64 39/61 2024-04-25T11:02:27.910 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:16.2.15-0.el8.x86_64 40/61 2024-04-25T11:02:27.911 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-mgr-2:16.2.15-0.el8.x86_64 40/61 2024-04-25T11:02:28.103 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:16.2.15-0.el8.x86_64 40/61 2024-04-25T11:02:28.103 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libcephsqlite-2:16.2.15-0.el8.x86_64 41/61 2024-04-25T11:02:28.346 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:16.2.15-0.el8.x86_64 41/61 2024-04-25T11:02:28.562 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados-devel-2:16.2.15-0.el8.x86_64 42/61 2024-04-25T11:02:28.570 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:16.2.15-0.el8.x86_64 43/61 2024-04-25T11:02:28.571 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-osd-2:16.2.15-0.el8.x86_64 43/61 2024-04-25T11:02:28.937 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:16.2.15-0.el8.x86_64 43/61 2024-04-25T11:02:28.943 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:16.2.15-0.el8.x86_64 44/61 2024-04-25T11:02:28.943 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-mds-2:16.2.15-0.el8.x86_64 44/61 2024-04-25T11:02:29.113 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:16.2.15-0.el8.x86_64 44/61 2024-04-25T11:02:29.118 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:16.2.15-0.el8.x86_64 45/61 2024-04-25T11:02:29.118 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-mon-2:16.2.15-0.el8.x86_64 45/61 2024-04-25T11:02:29.329 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:16.2.15-0.el8.x86_64 45/61 2024-04-25T11:02:29.329 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-selinux-2:16.2.15-0.el8.x86_64 46/61 2024-04-25T11:02:29.471 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:16.2.15-0.el8.x86_64 46/61 2024-04-25T11:02:29.494 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:16.2.15-0.el8.x86_64 47/61 2024-04-25T11:02:29.494 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-base-2:16.2.15-0.el8.x86_64 47/61 2024-04-25T11:02:29.823 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:16.2.15-0.el8.x86_64 47/61 2024-04-25T11:02:29.823 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-common-2:16.2.15-0.el8.x86_64 48/61 2024-04-25T11:02:30.080 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:16.2.15-0.el8.x86_64 48/61 2024-04-25T11:02:30.080 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libradosstriper1-2:16.2.15-0.el8.x86_64 49/61 2024-04-25T11:02:30.338 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:16.2.15-0.el8.x86_64 49/61 2024-04-25T11:02:30.512 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : python3-cephfs-2:16.2.15-0.el8.x86_64 50/61 2024-04-25T11:02:30.737 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : python3-rbd-2:16.2.15-0.el8.x86_64 51/61 2024-04-25T11:02:30.738 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.15-0.el8.x86_64 52/61 2024-04-25T11:02:30.987 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.15-0.el8.x86_64 52/61 2024-04-25T11:02:31.195 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : python3-rgw-2:16.2.15-0.el8.x86_64 53/61 2024-04-25T11:02:31.195 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librgw2-2:16.2.15-0.el8.x86_64 54/61 2024-04-25T11:02:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:16.2.15-0.el8.x86_64 54/61 2024-04-25T11:02:31.637 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : python3-rados-2:16.2.15-0.el8.x86_64 55/61 2024-04-25T11:02:31.637 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libcephfs2-2:16.2.15-0.el8.x86_64 56/61 2024-04-25T11:02:31.897 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:16.2.15-0.el8.x86_64 56/61 2024-04-25T11:02:32.062 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-fuse-2:16.2.15-0.el8.x86_64 57/61 2024-04-25T11:02:32.222 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : python3-ceph-argparse-2:16.2.15-0.el8.x86_64 58/61 2024-04-25T11:02:32.391 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : python3-ceph-common-2:16.2.15-0.el8.x86_64 59/61 2024-04-25T11:02:32.593 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 60/61 2024-04-25T11:02:32.594 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.15-0.el8.x86_64 61/61 2024-04-25T11:02:33.361 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.15-0.el8.x86_64 61/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 1/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.2.3-0.el8.x86_64 2/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:16.2.15-0.el8.x86_64 3/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.2.3-0.el8.x86_64 4/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:16.2.15-0.el8.x86_64 5/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.2.3-0.el8.x86_64 6/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:16.2.15-0.el8.x86_64 7/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.2.3-0.el8.x86_64 8/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:16.2.15-0.el8.x86_64 9/61 2024-04-25T11:02:33.362 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.2.3-0.el8.x86_64 10/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:16.2.15-0.el8.x86_64 11/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.2.3-0.el8.x86_64 12/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:16.2.15-0.el8.x86_64 13/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.2.3-0.el8.x86_64 14/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:16.2.15-0.el8.x86_64 15/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.2.3-0.el8.x86_64 16/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:16.2.15-0.el8.x86_64 17/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.2.3-0.el8.x86_64 18/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:16.2.15-0.el8.x86_64 19/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.2.3-0.el8.x86_64 20/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:16.2.15-0.el8.x86_64 21/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.2.3-0.el8.x86_64 22/61 2024-04-25T11:02:33.363 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:16.2.15-0.el8.x86_64 23/61 2024-04-25T11:02:33.364 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.2.3-0.el8.x86_64 24/61 2024-04-25T11:02:33.364 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:16.2.15-0.el8.x86_64 25/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.2.3-0.el8.x86_64 26/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:16.2.15-0.el8.x86_64 27/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.2.3-0.el8.x86_64 28/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:16.2.15-0.el8.x86_64 29/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.2.3-0.el8.x86_64 30/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:16.2.15-0.el8.x86_64 31/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.2.3-0.el8.x86_64 32/61 2024-04-25T11:02:33.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:16.2.15-0.el8.x86_64 33/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.2.3-0.el8.x86_64 34/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.15-0.el8.x86_64 35/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.2.3-0.el8.x86_64 36/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:16.2.15-0.el8.x86_64 37/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.2.3-0.el8.x86_64 38/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.15-0.el8.x86_64 39/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.2.3-0.el8.x86_64 40/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:16.2.15-0.el8.x86_64 41/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.2.3-0.el8.x86_64 42/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:16.2.15-0.el8.x86_64 43/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.2.3-0.el8.x86_64 44/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:16.2.15-0.el8.x86_64 45/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.2.3-0.el8.x86_64 46/61 2024-04-25T11:02:33.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:16.2.15-0.el8.x86_64 47/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.2.3-0.el8.x86_64 48/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:16.2.15-0.el8.x86_64 49/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.2.3-0.el8.x86_64 50/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:16.2.15-0.el8.x86_64 51/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.2.3-0.el8.x86_64 52/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:16.2.15-0.el8.x86_64 53/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.2.3-0.el8.x86_64 54/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:16.2.15-0.el8.x86_64 55/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.2.3-0.el8.x86_64 56/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:16.2.15-0.el8.x86_64 57/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.2.3-0.el8.x86_64 58/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:16.2.15-0.el8.x86_64 59/61 2024-04-25T11:02:33.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.2.3-0.el8.noarch 60/61 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 61/61 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.2.3-0.el8.noarch 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.2.3-0.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:34.802 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-25T11:02:34.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-04-25T11:02:35.264 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:02:35.288 INFO:teuthology.orchestra.run.smithi155.stdout:18.2.3-0.el8 2024-04-25T11:02:35.288 INFO:teuthology.packaging:The installed version of ceph is 18.2.3-0.el8 2024-04-25T11:02:35.289 INFO:teuthology.task.install:The correct ceph version 18.2.3-0 is installed. 2024-04-25T11:02:35.290 INFO:teuthology.run_tasks:Running task print... 2024-04-25T11:02:35.300 INFO:teuthology.task.print:**** done install.upgrade 2024-04-25T11:02:35.300 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-04-25T11:02:35.323 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:02:35.323 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:35.339 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:02:35.340 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T11:02:35.340 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T11:02:35.381 INFO:tasks.ceph.mon.a:Started 2024-04-25T11:02:35.382 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:02:35.382 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:35.393 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:02:35.393 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T11:02:35.393 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T11:02:35.436 INFO:tasks.ceph.mon.b:Started 2024-04-25T11:02:35.436 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:02:35.436 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:35.448 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:02:35.448 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T11:02:35.448 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T11:02:35.491 INFO:tasks.ceph.mon.c:Started 2024-04-25T11:02:35.491 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:02:35.491 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:35.510 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:02:35.510 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T11:02:35.511 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T11:02:35.552 INFO:tasks.ceph.mgr.x:Started 2024-04-25T11:02:35.552 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:02:35.553 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:35.584 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:02:35.584 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T11:02:35.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T11:02:35.627 INFO:tasks.ceph.mgr.y:Started 2024-04-25T11:02:35.628 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-04-25T11:02:35.636 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:02:35.637 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:02:35.637 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.767+0000 7fdf4ecb2700 1 Processor -- start 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.767+0000 7fdf4ecb2700 1 -- start start 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ecb2700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf4812c940 0x7fdf4812cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ecb2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf48071190 0x7fdf481348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ecb2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480722e0 con 0x7fdf480706a0 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ecb2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806c6e0 con 0x7fdf48071190 2024-04-25T11:02:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ecb2700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806d460 con 0x7fdf4812c940 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ca4e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf4812c940 0x7fdf4812cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf47fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fdf480706a0 0x7fdf48070a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47086/0 (socket says 172.21.15.155:47086) 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf47fff700 1 -- 172.21.15.155:0/785017664 learned_addr learned my addr 172.21.15.155:0/785017664 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4d24f700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf48071190 0x7fdf481348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf477fe700 1 -- 172.21.15.155:0/785017664 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120202871 0 0) 0x7fdf480722e0 con 0x7fdf480706a0 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf477fe700 1 -- 172.21.15.155:0/785017664 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf380035b0 con 0x7fdf480706a0 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4d24f700 1 -- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf4812c940 msgr2=0x7fdf4812cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4d24f700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf4812c940 0x7fdf4812cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4d24f700 1 -- 172.21.15.155:0/785017664 >> v1:172.21.15.155:6789/0 conn(0x7fdf480706a0 legacy=0x7fdf48070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4d24f700 1 -- 172.21.15.155:0/785017664 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48071c50 con 0x7fdf48071190 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4ca4e700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf4812c940 0x7fdf4812cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf4d24f700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf48071190 0x7fdf481348d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009dc0 tx=0x7fdf4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded0709711e18756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf477fe700 1 -- 172.21.15.155:0/785017664 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000e070 con 0x7fdf48071190 2024-04-25T11:02:35.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf477fe700 1 -- 172.21.15.155:0/785017664 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4000bb20 con 0x7fdf48071190 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.768+0000 7fdf477fe700 1 -- 172.21.15.155:0/785017664 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000be90 con 0x7fdf48071190 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf48071190 msgr2=0x7fdf481348d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf48071190 0x7fdf481348d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009dc0 tx=0x7fdf4000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/785017664 shutdown_connections 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf4812c940 0x7fdf4812cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/785017664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf48071190 0x7fdf481348d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/785017664 >> 172.21.15.155:0/785017664 conn(0x7fdf4809dbc0 msgr2=0x7fdf4809dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/785017664 shutdown_connections 2024-04-25T11:02:35.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.769+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/785017664 wait complete. 2024-04-25T11:02:35.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.770+0000 7fdf4ecb2700 1 Processor -- start 2024-04-25T11:02:35.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.770+0000 7fdf4ecb2700 1 -- start start 2024-04-25T11:02:35.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 0x7fdf4812e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf48071190 0x7fdf4812ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4812c940 0x7fdf481c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481318d0 con 0x7fdf4812c940 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4812f910 con 0x7fdf480706a0 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48130880 con 0x7fdf48071190 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 0x7fdf4812e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf47fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf48071190 0x7fdf4812ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 0x7fdf4812e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48228/0 (socket says 172.21.15.155:48228) 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4d24f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4812c940 0x7fdf481c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 -- 172.21.15.155:0/2485681670 learned_addr learned my addr 172.21.15.155:0/2485681670 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 -- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf48071190 msgr2=0x7fdf4812ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf48071190 0x7fdf4812ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 -- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4812c940 msgr2=0x7fdf481c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4812c940 0x7fdf481c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 -- 172.21.15.155:0/2485681670 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0094a0 con 0x7fdf480706a0 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf47fff700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf48071190 0x7fdf4812ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:35.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ca4e700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 0x7fdf4812e840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00ebe0 tx=0x7fdf3c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00c850 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c00c9b0 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.771+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf40009a70 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.772+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00cd40 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.772+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf48131cf0 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.772+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf48131030 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.772+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fdf3c013470 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.773+0000 7fdf47fff700 1 -- 172.21.15.155:0/2485681670 >> v1:172.21.15.155:6816/43298 conn(0x7fdf2c0421a0 legacy=0x7fdf2c044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.773+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fdf3c055970 con 0x7fdf480706a0 2024-04-25T11:02:35.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.776+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7fdf3c036200 con 0x7fdf480706a0 2024-04-25T11:02:35.954 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:35.953+0000 7f94bd147200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:02:35.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.957+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fdf48062510 con 0x7fdf480706a0 2024-04-25T11:02:35.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.959+0000 7fdf457fa700 1 -- 172.21.15.155:0/2485681670 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+97650 (secure 0 0 0) 0x7fdf3c017070 con 0x7fdf480706a0 2024-04-25T11:02:35.961 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 >> v1:172.21.15.155:6816/43298 conn(0x7fdf2c0421a0 legacy=0x7fdf2c044660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 msgr2=0x7fdf4812e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 0x7fdf4812e840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00ebe0 tx=0x7fdf3c00eef0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 shutdown_connections 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4812c940 0x7fdf481c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf480706a0 0x7fdf4812e840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 --2- 172.21.15.155:0/2485681670 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdf48071190 0x7fdf4812ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 >> 172.21.15.155:0/2485681670 conn(0x7fdf4809dbc0 msgr2=0x7fdf4812b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 shutdown_connections 2024-04-25T11:02:35.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:35.962+0000 7fdf4ecb2700 1 -- 172.21.15.155:0/2485681670 wait complete. 2024-04-25T11:02:35.980 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6816","nonce":43298}]},"active_addr":"172.21.15.155:6816/43298","active_change":"2024-04-25T11:00:53.384392+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":966131479}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":3077341019}]}]} 2024-04-25T11:02:35.983 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:02:35.983 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:02:35.983 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:36.017 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:36.017+0000 7f5ff79b3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:02:36.108 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.107+0000 7f1999913700 1 Processor -- start 2024-04-25T11:02:36.108 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.107+0000 7f1999913700 1 -- start start 2024-04-25T11:02:36.110 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1999913700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412c4f0 0x7f199412c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1999913700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 0x7f1994129140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1999913700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1994129680 0x7f1994129b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1999913700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19940969e0 con 0x7f1994129680 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1999913700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199406cb40 con 0x7f199412ce10 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1999913700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1994097420 con 0x7f199412c4f0 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f19927fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 0x7f1994129140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f19927fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 0x7f1994129140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48272/0 (socket says 172.21.15.155:48272) 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f19927fc700 1 -- 172.21.15.155:0/3377589774 learned_addr learned my addr 172.21.15.155:0/3377589774 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412c4f0 0x7f199412c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f19937fe700 1 --2- 172.21.15.155:0/3377589774 >> v2:172.21.15.155:3300/0 conn(0x7f1994129680 0x7f1994129b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 -- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 msgr2=0x7f1994129140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 0x7f1994129140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 -- 172.21.15.155:0/3377589774 >> v2:172.21.15.155:3300/0 conn(0x7f1994129680 msgr2=0x7f1994129b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 --2- 172.21.15.155:0/3377589774 >> v2:172.21.15.155:3300/0 conn(0x7f1994129680 0x7f1994129b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 -- 172.21.15.155:0/3377589774 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199406d140 con 0x7f199412c4f0 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f19927fc700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 0x7f1994129140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f19937fe700 1 --2- 172.21.15.155:0/3377589774 >> v2:172.21.15.155:3300/0 conn(0x7f1994129680 0x7f1994129b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.108+0000 7f1992ffd700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412c4f0 0x7f199412c8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1984009870 tx=0x7f1984009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88611a411c833139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1991ffb700 1 -- 172.21.15.155:0/3377589774 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984011830 con 0x7f199412c4f0 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1991ffb700 1 -- 172.21.15.155:0/3377589774 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984011990 con 0x7f199412c4f0 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1991ffb700 1 -- 172.21.15.155:0/3377589774 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984011d20 con 0x7f199412c4f0 2024-04-25T11:02:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 -- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412c4f0 msgr2=0x7f199412c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412c4f0 0x7f199412c8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1984009870 tx=0x7f1984009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 -- 172.21.15.155:0/3377589774 shutdown_connections 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 --2- 172.21.15.155:0/3377589774 >> v2:172.21.15.155:3300/0 conn(0x7f1994129680 0x7f1994129b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412c4f0 0x7f199412c8d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 --2- 172.21.15.155:0/3377589774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f199412ce10 0x7f1994129140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 -- 172.21.15.155:0/3377589774 >> 172.21.15.155:0/3377589774 conn(0x7f199409a140 msgr2=0x7f199406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 -- 172.21.15.155:0/3377589774 shutdown_connections 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.109+0000 7f1999913700 1 -- 172.21.15.155:0/3377589774 wait complete. 2024-04-25T11:02:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.110+0000 7f1999913700 1 Processor -- start 2024-04-25T11:02:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 -- start start 2024-04-25T11:02:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 0x7f19941220d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f199412c4f0 0x7f1994122610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412ce10 0x7f1994122b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994124b50 con 0x7f199412c4f0 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1994093790 con 0x7f1994129680 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1999913700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1994124eb0 con 0x7f199412ce10 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 0x7f19941220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f19927fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f199412c4f0 0x7f1994122610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 0x7f19941220d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48280/0 (socket says 172.21.15.155:48280) 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 -- 172.21.15.155:0/2344707762 learned_addr learned my addr 172.21.15.155:0/2344707762 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f19937fe700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412ce10 0x7f1994122b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 -- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412ce10 msgr2=0x7f1994122b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412ce10 0x7f1994122b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 -- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f199412c4f0 msgr2=0x7f1994122610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f199412c4f0 0x7f1994122610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f1992ffd700 1 -- 172.21.15.155:0/2344707762 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c009a70 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.111+0000 7f19927fc700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f199412c4f0 0x7f1994122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.112+0000 7f1992ffd700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 0x7f19941220d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1984000c00 tx=0x7f1984004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.112+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984004240 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.112+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198401f070 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.112+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988009550 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.112+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840045c0 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.112+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19840094a0 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.113+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1994062510 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.114+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f1984021030 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.114+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f1984056c40 con 0x7f1994129680 2024-04-25T11:02:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.114+0000 7f19927fc700 1 -- 172.21.15.155:0/2344707762 >> v1:172.21.15.155:6816/43298 conn(0x7f19800421d0 legacy=0x7f1980044690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.117+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f1984007230 con 0x7f1994129680 2024-04-25T11:02:36.220 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:36.220+0000 7f94bd147200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:02:36.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.283+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19941241e0 con 0x7f1994129680 2024-04-25T11:02:36.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.284+0000 7f197bfff700 1 -- 172.21.15.155:0/2344707762 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8597 (secure 0 0 0) 0x7f1984001c00 con 0x7f1994129680 2024-04-25T11:02:36.284 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:36.285 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":14,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:01:06.473410+0000","last_up_change":"2024-04-25T11:00:58.405672+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":43900}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":43900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":43900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":43900}]},"public_addr":"172.21.15.155:6800/43900","cluster_addr":"172.21.15.155:6801/43900","heartbeat_back_addr":"172.21.15.155:6803/43900","heartbeat_front_addr":"172.21.15.155:6802/43900","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:36.285 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:36.284+0000 7f5ff79b3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:02:36.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.285+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 >> v1:172.21.15.155:6816/43298 conn(0x7f19800421d0 legacy=0x7f1980044690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.285+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 msgr2=0x7f19941220d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.286 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.285+0000 7f1999913700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 0x7f19941220d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1984000c00 tx=0x7f1984004750 comp rx=0 tx=0).stop 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.285+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 shutdown_connections 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.285+0000 7f1999913700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f199412ce10 0x7f1994122b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.286+0000 7f1999913700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1994129680 0x7f19941220d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.286+0000 7f1999913700 1 --2- 172.21.15.155:0/2344707762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f199412c4f0 0x7f1994122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.286+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 >> 172.21.15.155:0/2344707762 conn(0x7f199409a140 msgr2=0x7f19941288e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.286+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 shutdown_connections 2024-04-25T11:02:36.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.286+0000 7f1999913700 1 -- 172.21.15.155:0/2344707762 wait complete. 2024-04-25T11:02:36.300 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:02:36.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:36.381 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:36.381+0000 7f94bd147200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:02:36.422 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:36.421+0000 7f5ff79b3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:02:36.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 Processor -- start 2024-04-25T11:02:36.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 -- start start 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f935806f3d0 0x7f9358133c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9358134170 0x7f9358136560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935806c740 con 0x7f935806e8e0 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9358097f90 con 0x7f935806f3d0 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.465+0000 7f9360187700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580700e0 con 0x7f9358134170 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f935806f3d0 0x7f9358133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f935806f3d0 0x7f9358133c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51744/0 (socket says 172.21.15.155:51744) 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 -- 172.21.15.155:0/2297488356 learned_addr learned my addr 172.21.15.155:0/2297488356 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935e724700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9358134170 0x7f9358136560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 -- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9358134170 msgr2=0x7f9358136560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9358134170 0x7f9358136560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 -- 172.21.15.155:0/2297488356 >> v1:172.21.15.155:6789/0 conn(0x7f935806e8e0 legacy=0x7f935806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935cf21700 1 -- 172.21.15.155:0/2297488356 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959750005 0 0) 0x7f9354001110 con 0x7f935806e8e0 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 -- 172.21.15.155:0/2297488356 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935809e7b0 con 0x7f935806f3d0 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935e724700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9358134170 0x7f9358136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935d722700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f935806f3d0 0x7f9358133c30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9348009920 tx=0x7f9348009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44aaf0b7529886ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935cf21700 1 -- 172.21.15.155:0/2297488356 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348011830 con 0x7f935806f3d0 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935cf21700 1 -- 172.21.15.155:0/2297488356 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348011990 con 0x7f935806f3d0 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.466+0000 7f935cf21700 1 -- 172.21.15.155:0/2297488356 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348011d20 con 0x7f935806f3d0 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 -- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f935806f3d0 msgr2=0x7f9358133c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f935806f3d0 0x7f9358133c30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9348009920 tx=0x7f9348009c30 comp rx=0 tx=0).stop 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 -- 172.21.15.155:0/2297488356 shutdown_connections 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9358134170 0x7f9358136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 --2- 172.21.15.155:0/2297488356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f935806f3d0 0x7f9358133c30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 -- 172.21.15.155:0/2297488356 >> 172.21.15.155:0/2297488356 conn(0x7f935809dc90 msgr2=0x7f935809e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 -- 172.21.15.155:0/2297488356 shutdown_connections 2024-04-25T11:02:36.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.467+0000 7f9360187700 1 -- 172.21.15.155:0/2297488356 wait complete. 2024-04-25T11:02:36.471 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.468+0000 7f9360187700 1 Processor -- start 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.468+0000 7f9360187700 1 -- start start 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f9360187700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 0x7f9358096100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f9360187700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f935806f3d0 0x7f9358096640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f9360187700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358134170 0x7f93581c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f9360187700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9358122c90 con 0x7f9358134170 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f9360187700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9358124900 con 0x7f935806e8e0 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f9360187700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93581245f0 con 0x7f935806f3d0 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 0x7f9358096100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935e724700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358134170 0x7f93581c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 0x7f9358096100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48304/0 (socket says 172.21.15.155:48304) 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 -- 172.21.15.155:0/398315151 learned_addr learned my addr 172.21.15.155:0/398315151 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 -- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f935806f3d0 msgr2=0x7f9358096640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f935806f3d0 0x7f9358096640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 -- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358134170 msgr2=0x7f93581c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358134170 0x7f93581c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935df23700 1 -- 172.21.15.155:0/398315151 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009a70 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.469+0000 7f935e724700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358134170 0x7f93581c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.470+0000 7f935df23700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 0x7f9358096100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f935400c780 tx=0x7f935400cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.470+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93540054e0 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.470+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354003680 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.470+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354003a30 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.470+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9348009550 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.470+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935809d6e0 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.471+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f9354003b90 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.471+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358123f50 con 0x7f935806e8e0 2024-04-25T11:02:36.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.471+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f935405c360 con 0x7f935806e8e0 2024-04-25T11:02:36.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.471+0000 7f935d722700 1 -- 172.21.15.155:0/398315151 >> v1:172.21.15.155:6816/43298 conn(0x7f933c0421a0 legacy=0x7f933c044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:36.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.475+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f9354055e60 con 0x7f935806e8e0 2024-04-25T11:02:36.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.627+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f935806c740 con 0x7f935806e8e0 2024-04-25T11:02:36.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.628+0000 7f934e7fc700 1 -- 172.21.15.155:0/398315151 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8597 (secure 0 0 0) 0x7f935806c740 con 0x7f935806e8e0 2024-04-25T11:02:36.628 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:36.629 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":14,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:01:06.473410+0000","last_up_change":"2024-04-25T11:00:58.405672+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":43900}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":43900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":43900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":43900}]},"public_addr":"172.21.15.155:6800/43900","cluster_addr":"172.21.15.155:6801/43900","heartbeat_back_addr":"172.21.15.155:6803/43900","heartbeat_front_addr":"172.21.15.155:6802/43900","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 >> v1:172.21.15.155:6816/43298 conn(0x7f933c0421a0 legacy=0x7f933c044660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 msgr2=0x7f9358096100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 0x7f9358096100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f935400c780 tx=0x7f935400cb40 comp rx=0 tx=0).stop 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 shutdown_connections 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358134170 0x7f93581c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f935806e8e0 0x7f9358096100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 --2- 172.21.15.155:0/398315151 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f935806f3d0 0x7f9358096640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 >> 172.21.15.155:0/398315151 conn(0x7f935809dc90 msgr2=0x7f935808f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 shutdown_connections 2024-04-25T11:02:36.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.630+0000 7f9360187700 1 -- 172.21.15.155:0/398315151 wait complete. 2024-04-25T11:02:36.644 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:02:36.644 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:02:36.645 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:02:36.645 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:02:36.713 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:36.711+0000 7f94bd147200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:02:36.756 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:36.756+0000 7f5ff79b3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:02:36.861 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:36.860+0000 7f94bd147200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:02:36.875 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.874+0000 7fce68709700 1 Processor -- start 2024-04-25T11:02:36.876 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.875+0000 7fce68709700 1 -- start start 2024-04-25T11:02:36.876 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.875+0000 7fce68709700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce60094d20 0x7fce60095100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.876 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.875+0000 7fce68709700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce60095a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.876 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.875+0000 7fce68709700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce600974f0 con 0x7fce6012f1d0 2024-04-25T11:02:36.876 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.875+0000 7fce68709700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6009ce50 con 0x7fce60095640 2024-04-25T11:02:36.876 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.875+0000 7fce68709700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce60096940 con 0x7fce60094d20 2024-04-25T11:02:36.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.876+0000 7fce65ca4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce60095a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.876+0000 7fce65ca4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce60095a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48306/0 (socket says 172.21.15.155:48306) 2024-04-25T11:02:36.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.876+0000 7fce65ca4700 1 -- 172.21.15.155:0/1809961216 learned_addr learned my addr 172.21.15.155:0/1809961216 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7fce65ca4700 1 -- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce60094d20 msgr2=0x7fce60095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7fce65ca4700 1 --2- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce60094d20 0x7fce60095100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7fce65ca4700 1 -- 172.21.15.155:0/1809961216 >> v1:172.21.15.155:6789/0 conn(0x7fce6012f1d0 legacy=0x7fce601315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7fce65ca4700 1 -- 172.21.15.155:0/1809961216 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60095fa0 con 0x7fce60095640 2024-04-25T11:02:36.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7fce65ca4700 1 --2- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce60095a60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009950 tx=0x7fce5c009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efae9ba5a2227654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.878+0000 7fce654a3700 1 -- 172.21.15.155:0/1809961216 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c009ea0 con 0x7fce60095640 2024-04-25T11:02:36.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.878+0000 7fce654a3700 1 -- 172.21.15.155:0/1809961216 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c004230 con 0x7fce60095640 2024-04-25T11:02:36.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.878+0000 7fce654a3700 1 -- 172.21.15.155:0/1809961216 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00b7e0 con 0x7fce60095640 2024-04-25T11:02:36.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.879+0000 7fce68709700 1 -- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 msgr2=0x7fce60095a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.879+0000 7fce68709700 1 --2- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce60095a60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009950 tx=0x7fce5c009c60 comp rx=0 tx=0).stop 2024-04-25T11:02:36.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 Processor -- start 2024-04-25T11:02:36.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 -- start start 2024-04-25T11:02:36.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 0x7f85a808eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f85a808f3e0 0x7f85a808b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a808b950 0x7f85a808bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a806e770 con 0x7f85a808f3e0 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a806ec70 con 0x7f85a806e390 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85ad617700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85a8072270 con 0x7f85a808b950 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85a6ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 0x7f85a808eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85a6ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 0x7f85a808eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48314/0 (socket says 172.21.15.155:48314) 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.877+0000 7f85a6ffd700 1 -- 172.21.15.155:0/286958693 learned_addr learned my addr 172.21.15.155:0/286958693 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a77fe700 1 --2- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a808b950 0x7f85a808bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f859ffff700 1 --2- 172.21.15.155:0/286958693 >> v2:172.21.15.155:3300/0 conn(0x7f85a808f3e0 0x7f85a808b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a6ffd700 1 -- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a808b950 msgr2=0x7f85a808bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a6ffd700 1 --2- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a808b950 0x7f85a808bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a6ffd700 1 -- 172.21.15.155:0/286958693 >> v2:172.21.15.155:3300/0 conn(0x7f85a808f3e0 msgr2=0x7f85a808b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a6ffd700 1 --2- 172.21.15.155:0/286958693 >> v2:172.21.15.155:3300/0 conn(0x7f85a808f3e0 0x7f85a808b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a6ffd700 1 -- 172.21.15.155:0/286958693 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a808ec00 con 0x7f85a806e390 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.880+0000 7f85a6ffd700 1 --2- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 0x7f85a808eea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85980088f0 tx=0x7f8598008c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7b159ef7a9ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7f85a67fc700 1 -- 172.21.15.155:0/286958693 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598014620 con 0x7f85a806e390 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7f85a67fc700 1 -- 172.21.15.155:0/286958693 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598010070 con 0x7f85a806e390 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7f85a67fc700 1 -- 172.21.15.155:0/286958693 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980149a0 con 0x7f85a806e390 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fa94a007700 1 Processor -- start 2024-04-25T11:02:36.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fce68709700 1 -- 172.21.15.155:0/1809961216 shutdown_connections 2024-04-25T11:02:36.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fce68709700 1 --2- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce60094d20 0x7fce60095100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fce68709700 1 --2- 172.21.15.155:0/1809961216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce60095a60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fce68709700 1 -- 172.21.15.155:0/1809961216 >> 172.21.15.155:0/1809961216 conn(0x7fce6009d480 msgr2=0x7fce6009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fa94a007700 1 -- start start 2024-04-25T11:02:36.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fce68709700 1 -- 172.21.15.155:0/1809961216 shutdown_connections 2024-04-25T11:02:36.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.881+0000 7fce68709700 1 -- 172.21.15.155:0/1809961216 wait complete. 2024-04-25T11:02:36.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa94a007700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa94412ac90 0x7fa94412b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa94a007700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94406f750 0x7fa94406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa94a007700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa94412b5b0 0x7fa94406de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa94a007700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94406c5b0 con 0x7fa94412ac90 2024-04-25T11:02:36.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa94a007700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94406fff0 con 0x7fa94406f750 2024-04-25T11:02:36.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa94a007700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa94406ba60 con 0x7fa94412b5b0 2024-04-25T11:02:36.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa94412ac90 0x7fa94412b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa94412ac90 0x7fa94412b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51758/0 (socket says 172.21.15.155:51758) 2024-04-25T11:02:36.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 -- 172.21.15.155:0/3113859754 learned_addr learned my addr 172.21.15.155:0/3113859754 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7f85ad617700 1 -- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 msgr2=0x7f85a808eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7f85ad617700 1 --2- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 0x7f85a808eea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85980088f0 tx=0x7f8598008c00 comp rx=0 tx=0).stop 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 -- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa94412b5b0 msgr2=0x7fa94406de70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa94412b5b0 0x7fa94406de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 -- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94406f750 msgr2=0x7fa94406d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94406f750 0x7fa94406d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.882+0000 7fa9437fe700 1 -- 172.21.15.155:0/3113859754 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94406e3b0 con 0x7fa94412ac90 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fa9437fe700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa94412b070 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa934007980 tx=0x7fa93400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f62799556b10456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 Processor -- start 2024-04-25T11:02:36.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fa942ffd700 1 -- 172.21.15.155:0/3113859754 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9340126d0 con 0x7fa94412ac90 2024-04-25T11:02:36.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 -- start start 2024-04-25T11:02:36.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fa942ffd700 1 -- 172.21.15.155:0/3113859754 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa934012830 con 0x7fa94412ac90 2024-04-25T11:02:36.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fa942ffd700 1 -- 172.21.15.155:0/3113859754 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934012be0 con 0x7fa94412ac90 2024-04-25T11:02:36.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce60094d20 0x7fce6012d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 -- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 msgr2=0x7fa94412b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa94412b070 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa934007980 tx=0x7fa93400d6b0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 -- 172.21.15.155:0/3113859754 shutdown_connections 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa94412b5b0 0x7fa94406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa94412b070 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 --2- 172.21.15.155:0/3113859754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94406f750 0x7fa94406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 -- 172.21.15.155:0/3113859754 >> 172.21.15.155:0/3113859754 conn(0x7fa94409c8d0 msgr2=0x7fa94409cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce6012dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce6012f1d0 0x7fce6008fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6012e7d0 con 0x7fce60094d20 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600927b0 con 0x7fce60095640 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.883+0000 7fce68709700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce601223c0 con 0x7fce6012f1d0 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce65ca4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce6012dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce65ca4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce6012dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48328/0 (socket says 172.21.15.155:48328) 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce65ca4700 1 -- 172.21.15.155:0/569047705 learned_addr learned my addr 172.21.15.155:0/569047705 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce664a5700 1 --2- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce60094d20 0x7fce6012d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce66ca6700 1 --2- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce6012f1d0 0x7fce6008fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 -- 172.21.15.155:0/3113859754 shutdown_connections 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fa94a007700 1 -- 172.21.15.155:0/3113859754 wait complete. 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7f85ad617700 1 -- 172.21.15.155:0/286958693 shutdown_connections 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7f85ad617700 1 --2- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a808b950 0x7f85a808bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7f85ad617700 1 --2- 172.21.15.155:0/286958693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a806e390 0x7f85a808eea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7f85ad617700 1 --2- 172.21.15.155:0/286958693 >> v2:172.21.15.155:3300/0 conn(0x7f85a808f3e0 0x7f85a808b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7f85ad617700 1 -- 172.21.15.155:0/286958693 >> 172.21.15.155:0/286958693 conn(0x7f85a809c8d0 msgr2=0x7f85a809cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce664a5700 1 -- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce6012f1d0 msgr2=0x7fce6008fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce664a5700 1 --2- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fce6012f1d0 0x7fce6008fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce664a5700 1 -- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 msgr2=0x7fce6012dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce664a5700 1 --2- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce60095640 0x7fce6012dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.884+0000 7fce664a5700 1 -- 172.21.15.155:0/569047705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c009550 con 0x7fce60094d20 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7f85ad617700 1 -- 172.21.15.155:0/286958693 shutdown_connections 2024-04-25T11:02:36.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7f85ad617700 1 -- 172.21.15.155:0/286958693 wait complete. 2024-04-25T11:02:36.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fce664a5700 1 --2- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce60094d20 0x7fce6012d9a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b450 tx=0x7fce5000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fa94a007700 1 Processor -- start 2024-04-25T11:02:36.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50011830 con 0x7fce60094d20 2024-04-25T11:02:36.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50011990 con 0x7fce60094d20 2024-04-25T11:02:36.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50011d40 con 0x7fce60094d20 2024-04-25T11:02:36.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fa94a007700 1 -- start start 2024-04-25T11:02:36.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce60091660 con 0x7fce60094d20 2024-04-25T11:02:36.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.885+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6012e420 con 0x7fce60094d20 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa94a007700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa944090b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa94a007700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94412b5b0 0x7fa9440910c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa94a007700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9440933e0 0x7fa944094250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa94a007700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa944091c70 con 0x7fa94412ac90 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa94a007700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa944094020 con 0x7fa94412b5b0 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa94a007700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa944093ae0 con 0x7fa9440933e0 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa944090b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa944090b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51778/0 (socket says 172.21.15.155:51778) 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 -- 172.21.15.155:0/911608837 learned_addr learned my addr 172.21.15.155:0/911608837 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa943fff700 1 --2- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9440933e0 0x7fa944094250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7f85ad617700 1 Processor -- start 2024-04-25T11:02:36.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 -- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9440933e0 msgr2=0x7fa944094250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 --2- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9440933e0 0x7fa944094250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 -- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94412b5b0 msgr2=0x7fa9440910c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 --2- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa94412b5b0 0x7fa9440910c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.886+0000 7fa9437fe700 1 -- 172.21.15.155:0/911608837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93400d050 con 0x7fa94412ac90 2024-04-25T11:02:36.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7fa9437fe700 1 --2- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa944090b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa93400dab0 tx=0x7fa934004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce60062510 con 0x7fce60094d20 2024-04-25T11:02:36.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934012070 con 0x7fa94412ac90 2024-04-25T11:02:36.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9340125e0 con 0x7fa94412ac90 2024-04-25T11:02:36.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934004260 con 0x7fa94412ac90 2024-04-25T11:02:36.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fce5001a400 con 0x7fce60094d20 2024-04-25T11:02:36.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fce500557b0 con 0x7fce60094d20 2024-04-25T11:02:36.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 -- start start 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a806e390 0x7f85a81e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85a808b950 0x7f85a81e1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a808f3e0 0x7f85a81e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a8090fb0 con 0x7f85a808b950 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a80912b0 con 0x7f85a808f3e0 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85ad617700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85a808a5f0 con 0x7f85a806e390 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85a6ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a806e390 0x7f85a81e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85a6ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a806e390 0x7f85a81e0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42160/0 (socket says 172.21.15.155:42160) 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85a6ffd700 1 -- 172.21.15.155:0/1938971617 learned_addr learned my addr 172.21.15.155:0/1938971617 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.887+0000 7f85a77fe700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a808f3e0 0x7f85a81e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f85a6ffd700 1 -- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a808f3e0 msgr2=0x7f85a81e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f85a6ffd700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85a808f3e0 0x7f85a81e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f85a6ffd700 1 -- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85a808b950 msgr2=0x7f85a81e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f859ffff700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85a808b950 0x7f85a81e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f85a6ffd700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85a808b950 0x7f85a81e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f85a6ffd700 1 -- 172.21.15.155:0/1938971617 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598008550 con 0x7f85a806e390 2024-04-25T11:02:36.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f859ffff700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85a808b950 0x7f85a81e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:36.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f85a6ffd700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a806e390 0x7f85a81e0b20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8598000c00 tx=0x7f8598004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 --> v1:172.21.15.155:6800/43900 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce4404c610 con 0x7fce44048920 2024-04-25T11:02:36.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7fce65ca4700 1 -- 172.21.15.155:0/569047705 >> v1:172.21.15.155:6816/43298 conn(0x7fce440421f0 legacy=0x7fce440446b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:36.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.889+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce50035f20 con 0x7fce60094d20 2024-04-25T11:02:36.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa944092560 con 0x7fa94412ac90 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa944094790 con 0x7fa94412ac90 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.889+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa94409d680 con 0x7fa94412ac90 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.888+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598014970 con 0x7f85a806e390 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.889+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598004040 con 0x7f85a806e390 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.889+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598004410 con 0x7f85a806e390 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.889+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a808aed0 con 0x7f85a806e390 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.889+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a8084090 con 0x7f85a806e390 2024-04-25T11:02:36.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.890+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f858c000fb0 con 0x7f85a806e390 2024-04-25T11:02:36.902 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 Processor -- start 2024-04-25T11:02:36.902 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f859801b050 con 0x7f85a806e390 2024-04-25T11:02:36.902 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fa9340043c0 con 0x7fa94412ac90 2024-04-25T11:02:36.902 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7f859ffff700 1 -- 172.21.15.155:0/1938971617 >> v1:172.21.15.155:6816/43298 conn(0x7f8594042340 legacy=0x7f8594044800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:36.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f8598056720 con 0x7f85a806e390 2024-04-25T11:02:36.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== osd.0 v1:172.21.15.155:6800/43900 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fce4404c610 con 0x7fce44048920 2024-04-25T11:02:36.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fa934057040 con 0x7fa94412ac90 2024-04-25T11:02:36.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 --> v1:172.21.15.155:6808/43899 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa93004c710 con 0x7fa930048a20 2024-04-25T11:02:36.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9340573e0 con 0x7fa94412ac90 2024-04-25T11:02:36.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fa93affd700 1 -- 172.21.15.155:0/911608837 >> v1:172.21.15.155:6816/43298 conn(0x7fa9300422f0 legacy=0x7fa9300447b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 --> v1:172.21.15.155:6812/43902 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f859404c760 con 0x7f8594048a70 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8598058cf0 con 0x7f85a806e390 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 -- start start 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c12b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc00c12b5b0 0x7fc00c06de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c06c5b0 con 0x7fc00c12b5b0 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c06fff0 con 0x7fc00c06f750 2024-04-25T11:02:36.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.891+0000 7fc0125df700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c06ba60 con 0x7fc00c12ac90 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc010b7c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc00c12b5b0 0x7fc00c06de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc010b7c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc00c12b5b0 0x7fc00c06de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51806/0 (socket says 172.21.15.155:51806) 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc010b7c700 1 -- 172.21.15.155:0/3309273778 learned_addr learned my addr 172.21.15.155:0/3309273778 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 --2- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c12b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 -- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 msgr2=0x7fc00c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 --2- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 -- 172.21.15.155:0/3309273778 >> v2:172.21.15.155:3300/0 conn(0x7fc00c12b5b0 msgr2=0x7fc00c06de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 --2- 172.21.15.155:0/3309273778 >> v2:172.21.15.155:3300/0 conn(0x7fc00c12b5b0 0x7fc00c06de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 -- 172.21.15.155:0/3309273778 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c06e3b0 con 0x7fc00c12ac90 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.892+0000 7fc00bfff700 1 --2- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c12b070 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc000009840 tx=0x7fc000009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5bdf503bd4ca6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.893+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== osd.2 v1:172.21.15.155:6812/43902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f859404c760 con 0x7f8594048a70 2024-04-25T11:02:36.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.894+0000 7fc00affd700 1 -- 172.21.15.155:0/3309273778 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000011830 con 0x7fc00c12ac90 2024-04-25T11:02:36.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.894+0000 7fc00affd700 1 -- 172.21.15.155:0/3309273778 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000011990 con 0x7fc00c12ac90 2024-04-25T11:02:36.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.894+0000 7fc00affd700 1 -- 172.21.15.155:0/3309273778 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000011d20 con 0x7fc00c12ac90 2024-04-25T11:02:36.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.894+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== osd.1 v1:172.21.15.155:6808/43899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fa93004c710 con 0x7fa930048a20 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 -- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 msgr2=0x7fc00c12b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 --2- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c12b070 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc000009840 tx=0x7fc000009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 -- 172.21.15.155:0/3309273778 shutdown_connections 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 --2- 172.21.15.155:0/3309273778 >> v2:172.21.15.155:3300/0 conn(0x7fc00c12b5b0 0x7fc00c06de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 --2- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c12b070 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 --2- 172.21.15.155:0/3309273778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.895+0000 7fc0125df700 1 -- 172.21.15.155:0/3309273778 >> 172.21.15.155:0/3309273778 conn(0x7fc00c09c8d0 msgr2=0x7fc00c09cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.900+0000 7fc0125df700 1 -- 172.21.15.155:0/3309273778 shutdown_connections 2024-04-25T11:02:36.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.900+0000 7fc0125df700 1 -- 172.21.15.155:0/3309273778 wait complete. 2024-04-25T11:02:36.907 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:36.901+0000 7f5ff79b3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.901+0000 7fc0125df700 1 Processor -- start 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.901+0000 7fc0125df700 1 -- start start 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc0125df700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c095aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc0125df700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c126740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc0125df700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c12b5b0 0x7fc00c126c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc0125df700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c095fe0 con 0x7fc00c12b5b0 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc0125df700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1289c0 con 0x7fc00c06f750 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc0125df700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c129270 con 0x7fc00c12ac90 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc00bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c095aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc00bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c095aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48348/0 (socket says 172.21.15.155:48348) 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc00bfff700 1 -- 172.21.15.155:0/4130008412 learned_addr learned my addr 172.21.15.155:0/4130008412 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:36.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.902+0000 7fc010b7c700 1 --2- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c12b5b0 0x7fc00c126c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.903+0000 7fc010b7c700 1 -- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 msgr2=0x7fc00c126740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.903+0000 7fc010b7c700 1 --2- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00c12ac90 0x7fc00c126740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.903+0000 7fc010b7c700 1 -- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 msgr2=0x7fc00c095aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.903+0000 7fc010b7c700 1 --2- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00c06f750 0x7fc00c095aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.903+0000 7fc010b7c700 1 -- 172.21.15.155:0/4130008412 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000094a0 con 0x7fc00c12b5b0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.903+0000 7fc010b7c700 1 --2- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c12b5b0 0x7fc00c126c80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc00c12ba00 tx=0x7fc00400b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.907+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004010070 con 0x7fc00c12b5b0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.907+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400bb40 con 0x7fc00c12b5b0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.907+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004003c40 con 0x7fc00c12b5b0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.907+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c12a070 con 0x7fc00c12b5b0 2024-04-25T11:02:36.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.907+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c1271c0 con 0x7fc00c12b5b0 2024-04-25T11:02:36.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.909+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fc004004030 con 0x7fc00c12b5b0 2024-04-25T11:02:36.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.910+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc004055840 con 0x7fc00c12b5b0 2024-04-25T11:02:36.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.910+0000 7fc00bfff700 1 -- 172.21.15.155:0/4130008412 >> v1:172.21.15.155:6816/43298 conn(0x7fbff80421f0 legacy=0x7fbff80446b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:36.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.909+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 --> v1:172.21.15.155:6800/43900 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce60133130 con 0x7fce44048920 2024-04-25T11:02:36.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.910+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 --> v1:172.21.15.155:6804/43901 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc00c1229a0 con 0x7fc00c1b3080 2024-04-25T11:02:36.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.910+0000 7fce56ffd700 1 -- 172.21.15.155:0/569047705 <== osd.0 v1:172.21.15.155:6800/43900 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1894478275) 0x7fce60133130 con 0x7fce44048920 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 >> v1:172.21.15.155:6800/43900 conn(0x7fce44048920 legacy=0x7fce4404ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 >> v1:172.21.15.155:6816/43298 conn(0x7fce440421f0 legacy=0x7fce440446b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce60094d20 msgr2=0x7fce6012d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce68709700 1 --2- 172.21.15.155:0/569047705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce60094d20 0x7fce6012d9a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b450 tx=0x7fce5000b760 comp rx=0 tx=0).stop 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce66ca6700 1 -- 172.21.15.155:0/569047705 reap_dead start 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 shutdown_connections 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.911+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 >> 172.21.15.155:0/569047705 conn(0x7fce6009d480 msgr2=0x7fce600983d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.912+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 shutdown_connections 2024-04-25T11:02:36.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.912+0000 7fce68709700 1 -- 172.21.15.155:0/569047705 wait complete. 2024-04-25T11:02:36.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.912+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== osd.3 v1:172.21.15.155:6804/43901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fc00c1229a0 con 0x7fc00c1b3080 2024-04-25T11:02:36.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.912+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 --> v1:172.21.15.155:6812/43902 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f858c002d00 con 0x7f8594048a70 2024-04-25T11:02:36.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f859f7fe700 1 -- 172.21.15.155:0/1938971617 <== osd.2 v1:172.21.15.155:6812/43902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 797704934) 0x7f858c002d00 con 0x7f8594048a70 2024-04-25T11:02:36.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 >> v1:172.21.15.155:6812/43902 conn(0x7f8594048a70 legacy=0x7f859404aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 >> v1:172.21.15.155:6816/43298 conn(0x7f8594042340 legacy=0x7f8594044800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a806e390 msgr2=0x7f85a81e0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85ad617700 1 --2- 172.21.15.155:0/1938971617 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f85a806e390 0x7f85a81e0b20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8598000c00 tx=0x7f8598004750 comp rx=0 tx=0).stop 2024-04-25T11:02:36.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85a77fe700 1 -- 172.21.15.155:0/1938971617 reap_dead start 2024-04-25T11:02:36.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 shutdown_connections 2024-04-25T11:02:36.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.913+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 >> 172.21.15.155:0/1938971617 conn(0x7f85a809c8d0 msgr2=0x7f85a808d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.919+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 shutdown_connections 2024-04-25T11:02:36.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.919+0000 7f85ad617700 1 -- 172.21.15.155:0/1938971617 wait complete. 2024-04-25T11:02:36.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.926+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 --> v1:172.21.15.155:6808/43899 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa944062510 con 0x7fa930048a20 2024-04-25T11:02:36.931 INFO:teuthology.orchestra.run.smithi155.stdout:38654705687 2024-04-25T11:02:36.931 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:02:36.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.931+0000 7fa940ff9700 1 -- 172.21.15.155:0/911608837 <== osd.1 v1:172.21.15.155:6808/43899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3722936805) 0x7fa944062510 con 0x7fa930048a20 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 >> v1:172.21.15.155:6808/43899 conn(0x7fa930048a20 legacy=0x7fa93004aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 >> v1:172.21.15.155:6816/43298 conn(0x7fa9300422f0 legacy=0x7fa9300447b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 msgr2=0x7fa944090b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 --2- 172.21.15.155:0/911608837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa94412ac90 0x7fa944090b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa93400dab0 tx=0x7fa934004b40 comp rx=0 tx=0).stop 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa943fff700 1 -- 172.21.15.155:0/911608837 reap_dead start 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 shutdown_connections 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 >> 172.21.15.155:0/911608837 conn(0x7fa94409c8d0 msgr2=0x7fa944098e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 shutdown_connections 2024-04-25T11:02:36.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.932+0000 7fa94a007700 1 -- 172.21.15.155:0/911608837 wait complete. 2024-04-25T11:02:36.935 INFO:teuthology.orchestra.run.smithi155.stdout:42949672983 2024-04-25T11:02:36.935 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:02:36.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.935+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 --> v1:172.21.15.155:6804/43901 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc00c062510 con 0x7fc00c1b3080 2024-04-25T11:02:36.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.935+0000 7fc008ff9700 1 -- 172.21.15.155:0/4130008412 <== osd.3 v1:172.21.15.155:6804/43901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1894478275) 0x7fc00c062510 con 0x7fc00c1b3080 2024-04-25T11:02:36.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 >> v1:172.21.15.155:6804/43901 conn(0x7fc00c1b3080 legacy=0x7fc00c061320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 >> v1:172.21.15.155:6816/43298 conn(0x7fbff80421f0 legacy=0x7fbff80446b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c12b5b0 msgr2=0x7fc00c126c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 --2- 172.21.15.155:0/4130008412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c12b5b0 0x7fc00c126c80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc00c12ba00 tx=0x7fc00400b320 comp rx=0 tx=0).stop 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc010b7c700 1 -- 172.21.15.155:0/4130008412 reap_dead start 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 shutdown_connections 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 >> 172.21.15.155:0/4130008412 conn(0x7fc00c09c8d0 msgr2=0x7fc00c1b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 shutdown_connections 2024-04-25T11:02:36.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:36.936+0000 7fc0125df700 1 -- 172.21.15.155:0/4130008412 wait complete. 2024-04-25T11:02:36.950 INFO:teuthology.orchestra.run.smithi155.stdout:42949672982 2024-04-25T11:02:36.950 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:02:36.955 INFO:teuthology.orchestra.run.smithi155.stdout:38654705687 2024-04-25T11:02:36.955 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:02:36.969 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:36.969+0000 7f94bd147200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:02:36.992 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:36.991+0000 7f5ff79b3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:02:37.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.087+0000 7fb512ba2700 1 Processor -- start 2024-04-25T11:02:37.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.087+0000 7fb512ba2700 1 -- start start 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb512ba2700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb50c135ea0 0x7fb50c138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb512ba2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c1387d0 0x7fb50c13abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb512ba2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c093f10 con 0x7fb50c093420 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb512ba2700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c096d30 con 0x7fb50c135ea0 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb512ba2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0960a0 con 0x7fb50c1387d0 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51093e700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb50c093420 0x7fb50c093800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47120/0 (socket says 172.21.15.155:47120) 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51093e700 1 -- 172.21.15.155:0/3075824743 learned_addr learned my addr 172.21.15.155:0/3075824743 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51113f700 1 --2- 172.21.15.155:0/3075824743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c1387d0 0x7fb50c13abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51113f700 1 -- 172.21.15.155:0/3075824743 >> v2:172.21.15.155:3300/0 conn(0x7fb50c135ea0 msgr2=0x7fb50c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51113f700 1 --2- 172.21.15.155:0/3075824743 >> v2:172.21.15.155:3300/0 conn(0x7fb50c135ea0 0x7fb50c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51113f700 1 -- 172.21.15.155:0/3075824743 >> v1:172.21.15.155:6789/0 conn(0x7fb50c093420 legacy=0x7fb50c093800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.088+0000 7fb51113f700 1 -- 172.21.15.155:0/3075824743 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c06ab70 con 0x7fb50c1387d0 2024-04-25T11:02:37.090 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.089+0000 7fb51113f700 1 --2- 172.21.15.155:0/3075824743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c1387d0 0x7fb50c13abc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb504009dc0 tx=0x7fb50400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b8ef53ffef7df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.089+0000 7fb50b7fe700 1 -- 172.21.15.155:0/3075824743 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50400e070 con 0x7fb50c1387d0 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.089+0000 7fb50b7fe700 1 -- 172.21.15.155:0/3075824743 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50400bb90 con 0x7fb50c1387d0 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.089+0000 7fb50b7fe700 1 -- 172.21.15.155:0/3075824743 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504012430 con 0x7fb50c1387d0 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 -- 172.21.15.155:0/3075824743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c1387d0 msgr2=0x7fb50c13abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 --2- 172.21.15.155:0/3075824743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c1387d0 0x7fb50c13abc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb504009dc0 tx=0x7fb50400b330 comp rx=0 tx=0).stop 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 -- 172.21.15.155:0/3075824743 shutdown_connections 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 --2- 172.21.15.155:0/3075824743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c1387d0 0x7fb50c13abc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 --2- 172.21.15.155:0/3075824743 >> v2:172.21.15.155:3300/0 conn(0x7fb50c135ea0 0x7fb50c138290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 -- 172.21.15.155:0/3075824743 >> 172.21.15.155:0/3075824743 conn(0x7fb50c09d480 msgr2=0x7fb50c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 -- 172.21.15.155:0/3075824743 shutdown_connections 2024-04-25T11:02:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.090+0000 7fb512ba2700 1 -- 172.21.15.155:0/3075824743 wait complete. 2024-04-25T11:02:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.091+0000 7fb512ba2700 1 Processor -- start 2024-04-25T11:02:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.091+0000 7fb512ba2700 1 -- start start 2024-04-25T11:02:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.091+0000 7fb512ba2700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb50c093420 0x7fb50c12c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb512ba2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 0x7fb50c12cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb512ba2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c1387d0 0x7fb50c12d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb512ba2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c12ba00 con 0x7fb50c1387d0 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb512ba2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c137cd0 con 0x7fb50c135ea0 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb512ba2700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c12bc70 con 0x7fb50c093420 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 0x7fb50c12cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 0x7fb50c12cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48376/0 (socket says 172.21.15.155:48376) 2024-04-25T11:02:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 -- 172.21.15.155:0/4091230575 learned_addr learned my addr 172.21.15.155:0/4091230575 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 -- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb50c093420 msgr2=0x7fb50c12c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb50c093420 0x7fb50c12c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 -- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c1387d0 msgr2=0x7fb50c12d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c1387d0 0x7fb50c12d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.092+0000 7fb50bfff700 1 -- 172.21.15.155:0/4091230575 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504009a70 con 0x7fb50c135ea0 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.093+0000 7fb50bfff700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 0x7fb50c12cd40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b520 tx=0x7fb4f800b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.093+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8011830 con 0x7fb50c135ea0 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.093+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f8011990 con 0x7fb50c135ea0 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.093+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c1355e0 con 0x7fb50c135ea0 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.093+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8011d40 con 0x7fb50c135ea0 2024-04-25T11:02:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.093+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c12e3b0 con 0x7fb50c135ea0 2024-04-25T11:02:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.094+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fb4f801a400 con 0x7fb50c135ea0 2024-04-25T11:02:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.095+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c12e690 con 0x7fb50c135ea0 2024-04-25T11:02:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.095+0000 7fb51093e700 1 -- 172.21.15.155:0/4091230575 >> v1:172.21.15.155:6816/43298 conn(0x7fb4fc0422a0 legacy=0x7fb4fc044760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.095+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fb4f8055910 con 0x7fb50c135ea0 2024-04-25T11:02:37.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.101+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7fb4f8035eb0 con 0x7fb50c135ea0 2024-04-25T11:02:37.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.120+0000 7fa0fb9a7700 1 Processor -- start 2024-04-25T11:02:37.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 -- start start 2024-04-25T11:02:37.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4093420 0x7fa0f4093800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0f4094980 0x7fa0f4094e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4095340 0x7fa0f4139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f4096d30 con 0x7fa0f4094980 2024-04-25T11:02:37.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40960a0 con 0x7fa0f4093420 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.121+0000 7fa0fb9a7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f4095c90 con 0x7fa0f4095340 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f9743700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4093420 0x7fa0f4093800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0f4094980 0x7fa0f4094e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f9743700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4093420 0x7fa0f4093800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48384/0 (socket says 172.21.15.155:48384) 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0f4094980 0x7fa0f4094e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51838/0 (socket says 172.21.15.155:51838) 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f9743700 1 -- 172.21.15.155:0/337036063 learned_addr learned my addr 172.21.15.155:0/337036063 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 -- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4095340 msgr2=0x7fa0f4139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4095340 0x7fa0f4139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 -- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4093420 msgr2=0x7fa0f4093800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4093420 0x7fa0f4093800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 -- 172.21.15.155:0/337036063 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f40957d0 con 0x7fa0f4094980 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.122+0000 7fa0f8f42700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4094980 0x7fa0f4094e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702d1809005748e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.123+0000 7fa0e7fff700 1 -- 172.21.15.155:0/337036063 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011830 con 0x7fa0f4094980 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.123+0000 7fa0e7fff700 1 -- 172.21.15.155:0/337036063 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0011990 con 0x7fa0f4094980 2024-04-25T11:02:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.123+0000 7fa0e7fff700 1 -- 172.21.15.155:0/337036063 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011d20 con 0x7fa0f4094980 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4094980 msgr2=0x7fa0f4094e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4094980 0x7fa0f4094e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f0009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/337036063 shutdown_connections 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4095340 0x7fa0f4139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4093420 0x7fa0f4093800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/337036063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4094980 0x7fa0f4094e00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/337036063 >> 172.21.15.155:0/337036063 conn(0x7fa0f409d480 msgr2=0x7fa0f409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/337036063 shutdown_connections 2024-04-25T11:02:37.125 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.124+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/337036063 wait complete. 2024-04-25T11:02:37.126 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.125+0000 7fa0fb9a7700 1 Processor -- start 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 -- start start 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4093420 0x7fa0f41c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4094980 0x7fa0f41c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 0x7fa0f41c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f4092790 con 0x7fa0f4095340 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41af9f0 con 0x7fa0f4094980 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0fb9a7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41b09f0 con 0x7fa0f4093420 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0f9f44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 0x7fa0f41c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0f9f44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 0x7fa0f41c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51854/0 (socket says 172.21.15.155:51854) 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.126+0000 7fa0f9f44700 1 -- 172.21.15.155:0/3424124260 learned_addr learned my addr 172.21.15.155:0/3424124260 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.127+0000 7fa0f8f42700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4094980 0x7fa0f41c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.127+0000 7fa0f9f44700 1 -- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4093420 msgr2=0x7fa0f41c4460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.127+0000 7fa0f9f44700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4093420 0x7fa0f41c4460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.127+0000 7fa0f9f44700 1 -- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4094980 msgr2=0x7fa0f41c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.127+0000 7fa0f9f44700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4094980 0x7fa0f41c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.127+0000 7fa0f9f44700 1 -- 172.21.15.155:0/3424124260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00094a0 con 0x7fa0f4095340 2024-04-25T11:02:37.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.128+0000 7fa0f9f44700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 0x7fa0f41c7780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b9a0 tx=0x7fa0ec00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.129+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00c930 con 0x7fa0f4095340 2024-04-25T11:02:37.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.129+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00cab0 con 0x7fa0f4095340 2024-04-25T11:02:37.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.129+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00ce60 con 0x7fa0f4095340 2024-04-25T11:02:37.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.130+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f41b0620 con 0x7fa0f4095340 2024-04-25T11:02:37.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.130+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f41c4ee0 con 0x7fa0f4095340 2024-04-25T11:02:37.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.131+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f409d860 con 0x7fa0f4095340 2024-04-25T11:02:37.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.131+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fa0ec01d3b0 con 0x7fa0f4095340 2024-04-25T11:02:37.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.132+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fa0ec006e40 con 0x7fa0f4095340 2024-04-25T11:02:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.134+0000 7fa0f9743700 1 -- 172.21.15.155:0/3424124260 >> v1:172.21.15.155:6816/43298 conn(0x7fa0d80421f0 legacy=0x7fa0d80446b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.135+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7fa0ec035ed0 con 0x7fa0f4095340 2024-04-25T11:02:37.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.164+0000 7f8e873c0700 1 Processor -- start 2024-04-25T11:02:37.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.164+0000 7f8e873c0700 1 -- start start 2024-04-25T11:02:37.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e873c0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e873c0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e80138750 0x7f8e8013ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e873c0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8008c500 con 0x7f8e80131dc0 2024-04-25T11:02:37.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e873c0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8008bf40 con 0x7f8e80138750 2024-04-25T11:02:37.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e873c0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801328b0 con 0x7f8e80135e20 2024-04-25T11:02:37.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e8595d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e80138750 0x7f8e8013ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e8595d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e80138750 0x7f8e8013ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51856/0 (socket says 172.21.15.155:51856) 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e8595d700 1 -- 172.21.15.155:0/287436710 learned_addr learned my addr 172.21.15.155:0/287436710 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.165+0000 7f8e8495b700 1 --2- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e8595d700 1 -- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 msgr2=0x7f8e80138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e8595d700 1 --2- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e8595d700 1 -- 172.21.15.155:0/287436710 >> v1:172.21.15.155:6789/0 conn(0x7f8e80131dc0 legacy=0x7f8e801321a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e8595d700 1 -- 172.21.15.155:0/287436710 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80097e90 con 0x7f8e80138750 2024-04-25T11:02:37.167 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e8595d700 1 --2- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8013ab40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009e50 tx=0x7f8e7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30258f47232d7435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e77fff700 1 -- 172.21.15.155:0/287436710 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800e070 con 0x7f8e80138750 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e77fff700 1 -- 172.21.15.155:0/287436710 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e78003a20 con 0x7f8e80138750 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.166+0000 7f8e77fff700 1 -- 172.21.15.155:0/287436710 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78003dd0 con 0x7f8e80138750 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.167+0000 7f8e873c0700 1 -- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 msgr2=0x7f8e8013ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.167+0000 7f8e873c0700 1 --2- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8013ab40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009e50 tx=0x7f8e7800b330 comp rx=0 tx=0).stop 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.167+0000 7f8e873c0700 1 -- 172.21.15.155:0/287436710 shutdown_connections 2024-04-25T11:02:37.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.167+0000 7f8e873c0700 1 --2- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8013ab40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.167+0000 7f8e873c0700 1 --2- 172.21.15.155:0/287436710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.167+0000 7f8e873c0700 1 -- 172.21.15.155:0/287436710 >> 172.21.15.155:0/287436710 conn(0x7f8e8009dac0 msgr2=0x7f8e8009ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.168+0000 7f8e873c0700 1 -- 172.21.15.155:0/287436710 shutdown_connections 2024-04-25T11:02:37.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.168+0000 7f8e873c0700 1 -- 172.21.15.155:0/287436710 wait complete. 2024-04-25T11:02:37.169 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.168+0000 7f5ff79b3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:02:37.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.169+0000 7f8e873c0700 1 Processor -- start 2024-04-25T11:02:37.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.169+0000 7f8e873c0700 1 -- start start 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.169+0000 7f8e873c0700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e80131dc0 0x7f8e80071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.169+0000 7f8e873c0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e873c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8006e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e873c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800707c0 con 0x7f8e80138750 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e873c0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801355a0 con 0x7f8e80135e20 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e873c0700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8008e600 con 0x7f8e80131dc0 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8595d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8595d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8006e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51866/0 (socket says 172.21.15.155:51866) 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8595d700 1 -- 172.21.15.155:0/674690210 learned_addr learned my addr 172.21.15.155:0/674690210 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8495b700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8495b700 1 -- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e80131dc0 msgr2=0x7f8e80071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8495b700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e80131dc0 0x7f8e80071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8495b700 1 -- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 msgr2=0x7f8e8006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.172 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8495b700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.172 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.170+0000 7f8e8495b700 1 -- 172.21.15.155:0/674690210 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009b30 con 0x7f8e80135e20 2024-04-25T11:02:37.172 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.171+0000 7f8e8495b700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80071fa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e930 tx=0x7f8e7c00ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.173 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.171+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00ee80 con 0x7f8e80135e20 2024-04-25T11:02:37.173 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.172+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c00c850 con 0x7f8e80135e20 2024-04-25T11:02:37.173 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.172+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00cbc0 con 0x7f8e80135e20 2024-04-25T11:02:37.173 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.172+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8006f600 con 0x7f8e80135e20 2024-04-25T11:02:37.173 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.172+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8006e940 con 0x7f8e80135e20 2024-04-25T11:02:37.173 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.173+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e80062510 con 0x7f8e80135e20 2024-04-25T11:02:37.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.174+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f8e7c00cd20 con 0x7f8e80135e20 2024-04-25T11:02:37.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.174+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f8e7c056220 con 0x7f8e80135e20 2024-04-25T11:02:37.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.174+0000 7f8e8515c700 1 -- 172.21.15.155:0/674690210 >> v1:172.21.15.155:6816/43298 conn(0x7f8e640421f0 legacy=0x7f8e640446b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.178+0000 7f26f7817700 1 Processor -- start 2024-04-25T11:02:37.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.178+0000 7f26f7817700 1 -- start start 2024-04-25T11:02:37.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f7817700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071390 0x7f26f0071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f7817700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26f0071cb0 0x7f26f00960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f7817700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0096aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f7817700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f006dfb0 con 0x7f26f0071cb0 2024-04-25T11:02:37.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f7817700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f009d3f0 con 0x7f26f0071390 2024-04-25T11:02:37.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f7817700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26f006ee90 con 0x7f26f00965f0 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0096aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0096aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42222/0 (socket says 172.21.15.155:42222) 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 -- 172.21.15.155:0/334667858 learned_addr learned my addr 172.21.15.155:0/334667858 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 -- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071390 msgr2=0x7f26f0071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 --2- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071390 0x7f26f0071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 -- 172.21.15.155:0/334667858 >> v2:172.21.15.155:3300/0 conn(0x7f26f0071cb0 msgr2=0x7f26f00960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 --2- 172.21.15.155:0/334667858 >> v2:172.21.15.155:3300/0 conn(0x7f26f0071cb0 0x7f26f00960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.179+0000 7f26f5db4700 1 -- 172.21.15.155:0/334667858 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f009d620 con 0x7f26f00965f0 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.180+0000 7f26f5db4700 1 --2- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0096aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009e80 tx=0x7f26e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a175d12af619e897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.180+0000 7f26e7fff700 1 -- 172.21.15.155:0/334667858 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800e070 con 0x7f26f00965f0 2024-04-25T11:02:37.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.180+0000 7f26e7fff700 1 -- 172.21.15.155:0/334667858 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800b980 con 0x7f26f00965f0 2024-04-25T11:02:37.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.180+0000 7f26e7fff700 1 -- 172.21.15.155:0/334667858 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800bd30 con 0x7f26f00965f0 2024-04-25T11:02:37.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.181+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f8e7c036af0 con 0x7f8e80135e20 2024-04-25T11:02:37.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.181+0000 7f26f7817700 1 -- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 msgr2=0x7f26f0096aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.181+0000 7f26f7817700 1 --2- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0096aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009e80 tx=0x7f26e800b330 comp rx=0 tx=0).stop 2024-04-25T11:02:37.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.183+0000 7f26f7817700 1 -- 172.21.15.155:0/334667858 shutdown_connections 2024-04-25T11:02:37.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.183+0000 7f26f7817700 1 --2- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0096aa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.183+0000 7f26f7817700 1 --2- 172.21.15.155:0/334667858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071390 0x7f26f0071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.183+0000 7f26f7817700 1 --2- 172.21.15.155:0/334667858 >> v2:172.21.15.155:3300/0 conn(0x7f26f0071cb0 0x7f26f00960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.183+0000 7f26f7817700 1 -- 172.21.15.155:0/334667858 >> 172.21.15.155:0/334667858 conn(0x7f26f009c8d0 msgr2=0x7f26f009cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.185 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.185+0000 7f94bd147200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:02:37.186 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.186+0000 7f26f7817700 1 -- 172.21.15.155:0/334667858 shutdown_connections 2024-04-25T11:02:37.186 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.186+0000 7f26f7817700 1 -- 172.21.15.155:0/334667858 wait complete. 2024-04-25T11:02:37.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.187+0000 7f26f7817700 1 Processor -- start 2024-04-25T11:02:37.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.187+0000 7f26f7817700 1 -- start start 2024-04-25T11:02:37.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.188+0000 7f26f7817700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f0071390 0x7f26f0133420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.188+0000 7f26f7817700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 0x7f26f0133960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.188+0000 7f26f7817700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.188+0000 7f26f7817700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0126640 con 0x7f26f0071390 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.188+0000 7f26f7817700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0136530 con 0x7f26f0071cb0 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.188+0000 7f26f7817700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0134f80 con 0x7f26f00965f0 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 0x7f26f0133960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 0x7f26f0133960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48430/0 (socket says 172.21.15.155:48430) 2024-04-25T11:02:37.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 -- 172.21.15.155:0/1511208006 learned_addr learned my addr 172.21.15.155:0/1511208006 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:37.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 -- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 msgr2=0x7f26f0133ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:37.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0133ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 -- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f0071390 msgr2=0x7f26f0133420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f0071390 0x7f26f0133420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.189+0000 7f26f4db2700 1 -- 172.21.15.155:0/1511208006 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8009b30 con 0x7f26f0071cb0 2024-04-25T11:02:37.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.190+0000 7f26f4db2700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 0x7f26f0133960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b500 tx=0x7f26ec00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:37.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.190+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec011830 con 0x7f26f0071cb0 2024-04-25T11:02:37.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.190+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f01359f0 con 0x7f26f0071cb0 2024-04-25T11:02:37.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.190+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec0119b0 con 0x7f26f0071cb0 2024-04-25T11:02:37.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.190+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec011d40 con 0x7f26f0071cb0 2024-04-25T11:02:37.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.190+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f01cdd90 con 0x7f26f0071cb0 2024-04-25T11:02:37.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.192+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f0128610 con 0x7f26f0071cb0 2024-04-25T11:02:37.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.193+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f26ec01a400 con 0x7f26f0071cb0 2024-04-25T11:02:37.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.196+0000 7f26f55b3700 1 -- 172.21.15.155:0/1511208006 >> v1:172.21.15.155:6816/43298 conn(0x7f26d40421a0 legacy=0x7f26d4044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.196+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f26ec037250 con 0x7f26f0071cb0 2024-04-25T11:02:37.200 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.199+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f26ec07fc20 con 0x7f26f0071cb0 2024-04-25T11:02:37.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.274+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb50c062510 con 0x7fb50c135ea0 2024-04-25T11:02:37.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.275+0000 7fb5097fa700 1 -- 172.21.15.155:0/4091230575 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb4f80396a0 con 0x7fb50c135ea0 2024-04-25T11:02:37.276 INFO:teuthology.orchestra.run.smithi155.stdout:42949672981 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 >> v1:172.21.15.155:6816/43298 conn(0x7fb4fc0422a0 legacy=0x7fb4fc044760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 msgr2=0x7fb50c12cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 0x7fb50c12cd40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b520 tx=0x7fb4f800b8e0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 shutdown_connections 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c1387d0 0x7fb50c12d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb50c093420 0x7fb50c12c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 --2- 172.21.15.155:0/4091230575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb50c135ea0 0x7fb50c12cd40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 >> 172.21.15.155:0/4091230575 conn(0x7fb50c09d480 msgr2=0x7fb50c130ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 shutdown_connections 2024-04-25T11:02:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.277+0000 7fb512ba2700 1 -- 172.21.15.155:0/4091230575 wait complete. 2024-04-25T11:02:37.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672983 got 42949672981 for osd.2 2024-04-25T11:02:37.311 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.310+0000 7f5ff79b3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:02:37.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.335+0000 7fa0f9743700 1 -- 172.21.15.155:0/3424124260 >> v1:172.21.15.155:6816/43298 conn(0x7fa0d80421f0 legacy=0x7fa0d80446b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.362 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.361+0000 7f94bd147200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:02:37.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.375+0000 7f8e8515c700 1 -- 172.21.15.155:0/674690210 >> v1:172.21.15.155:6816/43298 conn(0x7f8e640421f0 legacy=0x7f8e640446b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.397+0000 7f26f55b3700 1 -- 172.21.15.155:0/1511208006 >> v1:172.21.15.155:6816/43298 conn(0x7f26d40421a0 legacy=0x7f26d4044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:37.401 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.400+0000 7f5ff79b3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:02:37.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.407+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa0f4062510 con 0x7fa0f4095340 2024-04-25T11:02:37.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.407+0000 7fa0e5ffb700 1 -- 172.21.15.155:0/3424124260 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0ec019070 con 0x7fa0f4095340 2024-04-25T11:02:37.408 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 >> v1:172.21.15.155:6816/43298 conn(0x7fa0d80421f0 legacy=0x7fa0d80446b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 msgr2=0x7fa0f41c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 0x7fa0f41c7780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b9a0 tx=0x7fa0ec00bcb0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 shutdown_connections 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0f4095340 0x7fa0f41c7780 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa0f4093420 0x7fa0f41c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 --2- 172.21.15.155:0/3424124260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0f4094980 0x7fa0f41c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 >> 172.21.15.155:0/3424124260 conn(0x7fa0f409d480 msgr2=0x7fa0f4070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 shutdown_connections 2024-04-25T11:02:37.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.410+0000 7fa0fb9a7700 1 -- 172.21.15.155:0/3424124260 wait complete. 2024-04-25T11:02:37.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.414+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f26f0062510 con 0x7f26f0071cb0 2024-04-25T11:02:37.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.414+0000 7f26e5ffb700 1 -- 172.21.15.155:0/1511208006 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26ec015070 con 0x7f26f0071cb0 2024-04-25T11:02:37.415 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 >> v1:172.21.15.155:6816/43298 conn(0x7f26d40421a0 legacy=0x7f26d4044660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 msgr2=0x7f26f0133960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 0x7f26f0133960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b500 tx=0x7f26ec00b8c0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 shutdown_connections 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f26f00965f0 0x7f26f0133ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f0071390 0x7f26f0133420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 --2- 172.21.15.155:0/1511208006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f0071cb0 0x7f26f0133960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 >> 172.21.15.155:0/1511208006 conn(0x7f26f009c8d0 msgr2=0x7f26f0132a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 shutdown_connections 2024-04-25T11:02:37.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.416+0000 7f26f7817700 1 -- 172.21.15.155:0/1511208006 wait complete. 2024-04-25T11:02:37.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.3 2024-04-25T11:02:37.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.0 2024-04-25T11:02:37.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.458+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e8008e3e0 con 0x7f8e80135e20 2024-04-25T11:02:37.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.458+0000 7f8e75ffb700 1 -- 172.21.15.155:0/674690210 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e7c017030 con 0x7f8e80135e20 2024-04-25T11:02:37.459 INFO:teuthology.orchestra.run.smithi155.stdout:42949672980 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 >> v1:172.21.15.155:6816/43298 conn(0x7f8e640421f0 legacy=0x7f8e640446b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 msgr2=0x7f8e80071fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80071fa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e930 tx=0x7f8e7c00ec40 comp rx=0 tx=0).stop 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 shutdown_connections 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e80138750 0x7f8e8006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e80131dc0 0x7f8e80071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 --2- 172.21.15.155:0/674690210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e80135e20 0x7f8e80071fa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 >> 172.21.15.155:0/674690210 conn(0x7f8e8009dac0 msgr2=0x7f8e8006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 shutdown_connections 2024-04-25T11:02:37.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:37.460+0000 7f8e873c0700 1 -- 172.21.15.155:0/674690210 wait complete. 2024-04-25T11:02:37.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672982 got 42949672980 for osd.1 2024-04-25T11:02:37.473 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.472+0000 7f94bd147200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:02:37.505 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.505+0000 7f5ff79b3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:02:37.551 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.551+0000 7f94bd147200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:02:37.584 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.584+0000 7f5ff79b3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:02:37.630 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.630+0000 7f94bd147200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:02:37.773 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.773+0000 7f5ff79b3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:02:37.787 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.787+0000 7f94bd147200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:02:37.952 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:37.952+0000 7f5ff79b3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:02:37.953 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:37.952+0000 7f94bd147200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:02:38.036 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:38.035+0000 7f94bd147200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:02:38.036 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:38.036+0000 7f5ff79b3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:02:38.293 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:02:38.307 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:38.307+0000 7f94bd147200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:02:38.307 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:38.307+0000 7f5ff79b3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:02:38.382 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:38.382+0000 7f94bd147200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:02:38.383 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:38.382+0000 7f5ff79b3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.418+0000 7f3a2d8bf700 1 Processor -- start 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.418+0000 7f3a2d8bf700 1 -- start start 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.418+0000 7f3a2d8bf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a2806eef0 0x7f3a28133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.418+0000 7f3a2d8bf700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3a281341e0 0x7f3a281365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a2d8bf700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a28072780 con 0x7f3a2806e400 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a2d8bf700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2806ca20 con 0x7f3a281341e0 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a2d8bf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2806fb80 con 0x7f3a2806eef0 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a2806eef0 0x7f3a28133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a26ffd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3a2806e400 0x7f3a2806e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47146/0 (socket says 172.21.15.155:47146) 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a26ffd700 1 -- 172.21.15.155:0/2004146002 learned_addr learned my addr 172.21.15.155:0/2004146002 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a277fe700 1 --2- 172.21.15.155:0/2004146002 >> v2:172.21.15.155:3300/0 conn(0x7f3a281341e0 0x7f3a281365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a267fc700 1 -- 172.21.15.155:0/2004146002 >> v2:172.21.15.155:3300/0 conn(0x7f3a281341e0 msgr2=0x7f3a281365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a267fc700 1 --2- 172.21.15.155:0/2004146002 >> v2:172.21.15.155:3300/0 conn(0x7f3a281341e0 0x7f3a281365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a267fc700 1 -- 172.21.15.155:0/2004146002 >> v1:172.21.15.155:6789/0 conn(0x7f3a2806e400 legacy=0x7f3a2806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a267fc700 1 -- 172.21.15.155:0/2004146002 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2806c6e0 con 0x7f3a2806eef0 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a25ffb700 1 -- 172.21.15.155:0/2004146002 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540609154 0 0) 0x7f3a28072780 con 0x7f3a2806e400 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a267fc700 1 --2- 172.21.15.155:0/2004146002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a2806eef0 0x7f3a28133ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009870 tx=0x7f3a1c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c1e514d045f3bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a25ffb700 1 -- 172.21.15.155:0/2004146002 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c011830 con 0x7f3a2806eef0 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a25ffb700 1 -- 172.21.15.155:0/2004146002 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c011990 con 0x7f3a2806eef0 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.419+0000 7f3a25ffb700 1 -- 172.21.15.155:0/2004146002 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c011d20 con 0x7f3a2806eef0 2024-04-25T11:02:38.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/2004146002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a2806eef0 msgr2=0x7f3a28133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/2004146002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a2806eef0 0x7f3a28133ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009870 tx=0x7f3a1c009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/2004146002 shutdown_connections 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/2004146002 >> v2:172.21.15.155:3300/0 conn(0x7f3a281341e0 0x7f3a281365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/2004146002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a2806eef0 0x7f3a28133ca0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/2004146002 >> 172.21.15.155:0/2004146002 conn(0x7f3a2809dac0 msgr2=0x7f3a2809ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/2004146002 shutdown_connections 2024-04-25T11:02:38.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.420+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/2004146002 wait complete. 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.421+0000 7f3a2d8bf700 1 Processor -- start 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.421+0000 7f3a2d8bf700 1 -- start start 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a2d8bf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 0x7f3a2812ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a2d8bf700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 0x7f3a2812bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a2d8bf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a281341e0 0x7f3a2812c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a2d8bf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28133d20 con 0x7f3a2806e400 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a2d8bf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28133610 con 0x7f3a281341e0 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a2d8bf700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280913f0 con 0x7f3a2806eef0 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a26ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 0x7f3a2812ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 0x7f3a2812bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a26ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 0x7f3a2812ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51886/0 (socket says 172.21.15.155:51886) 2024-04-25T11:02:38.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 0x7f3a2812bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42226/0 (socket says 172.21.15.155:42226) 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a26ffd700 1 -- 172.21.15.155:0/120016562 learned_addr learned my addr 172.21.15.155:0/120016562 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.422+0000 7f3a277fe700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a281341e0 0x7f3a2812c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a26ffd700 1 -- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 msgr2=0x7f3a2812bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a26ffd700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 0x7f3a2812bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a26ffd700 1 -- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a281341e0 msgr2=0x7f3a2812c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a26ffd700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a281341e0 0x7f3a2812c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a26ffd700 1 -- 172.21.15.155:0/120016562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0094a0 con 0x7f3a2806e400 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a267fc700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 0x7f3a2812bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a26ffd700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 0x7f3a2812ba30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c520 tx=0x7f3a1800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18004c80 con 0x7f3a2806e400 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18004e00 con 0x7f3a2806e400 2024-04-25T11:02:38.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18007310 con 0x7f3a2806e400 2024-04-25T11:02:38.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a28090ca0 con 0x7f3a2806e400 2024-04-25T11:02:38.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.423+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2808f410 con 0x7f3a2806e400 2024-04-25T11:02:38.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.424+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a28091d80 con 0x7f3a2806e400 2024-04-25T11:02:38.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.425+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f3a18007470 con 0x7f3a2806e400 2024-04-25T11:02:38.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.425+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f3a18055c20 con 0x7f3a2806e400 2024-04-25T11:02:38.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.425+0000 7f3a267fc700 1 -- 172.21.15.155:0/120016562 >> v1:172.21.15.155:6816/43298 conn(0x7f3a140421a0 legacy=0x7f3a14044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:38.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.428+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f3a18036220 con 0x7f3a2806e400 2024-04-25T11:02:38.430 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:02:38.432 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:02:38.474 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:02:38.480 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:38.480+0000 7f94bd147200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:02:38.480 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:38.479+0000 7f5ff79b3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:02:38.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.569+0000 7f7ee3980700 1 Processor -- start 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.569+0000 7f7ee3980700 1 -- start start 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee3980700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc12e590 0x7f7edc130980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee3980700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc130ec0 0x7f7edc1332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee3980700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0728f0 con 0x7f7edc1282e0 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee3980700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc06f960 con 0x7f7edc12e590 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee3980700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc06fe90 con 0x7f7edc130ec0 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee171c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7edc1282e0 0x7f7edc1286c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47150/0 (socket says 172.21.15.155:47150) 2024-04-25T11:02:38.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.570+0000 7f7ee171c700 1 -- 172.21.15.155:0/4161840460 learned_addr learned my addr 172.21.15.155:0/4161840460 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:38.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ee1f1d700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc130ec0 0x7f7edc1332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ee0f1b700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc12e590 0x7f7edc130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ed3fff700 1 -- 172.21.15.155:0/4161840460 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891347195 0 0) 0x7f7edc0728f0 con 0x7f7edc1282e0 2024-04-25T11:02:38.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ed3fff700 1 -- 172.21.15.155:0/4161840460 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec80035b0 con 0x7f7edc1282e0 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc130ec0 msgr2=0x7f7edc1332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ee0f1b700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc130ec0 0x7f7edc1332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/4161840460 >> v1:172.21.15.155:6789/0 conn(0x7f7edc1282e0 legacy=0x7f7edc1286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.571+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/4161840460 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc071060 con 0x7f7edc12e590 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.572+0000 7f7ee0f1b700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc12e590 0x7f7edc130980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800dc90 tx=0x7f7ed800f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4dfcd3e8ebbba69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.572+0000 7f7ed3fff700 1 -- 172.21.15.155:0/4161840460 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8012070 con 0x7f7edc12e590 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.572+0000 7f7ed3fff700 1 -- 172.21.15.155:0/4161840460 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed800d2e0 con 0x7f7edc12e590 2024-04-25T11:02:38.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.572+0000 7f7ed3fff700 1 -- 172.21.15.155:0/4161840460 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8004230 con 0x7f7edc12e590 2024-04-25T11:02:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.573+0000 7f7ee3980700 1 -- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc12e590 msgr2=0x7f7edc130980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.573+0000 7f7ee3980700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc12e590 0x7f7edc130980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800dc90 tx=0x7f7ed800f050 comp rx=0 tx=0).stop 2024-04-25T11:02:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.573+0000 7f7ee3980700 1 -- 172.21.15.155:0/4161840460 shutdown_connections 2024-04-25T11:02:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.573+0000 7f7ee3980700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc130ec0 0x7f7edc1332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.573+0000 7f7ee3980700 1 --2- 172.21.15.155:0/4161840460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc12e590 0x7f7edc130980 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.573+0000 7f7ee3980700 1 -- 172.21.15.155:0/4161840460 >> 172.21.15.155:0/4161840460 conn(0x7f7edc09dac0 msgr2=0x7f7edc09ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.574+0000 7f7ee3980700 1 -- 172.21.15.155:0/4161840460 shutdown_connections 2024-04-25T11:02:38.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.574+0000 7f7ee3980700 1 -- 172.21.15.155:0/4161840460 wait complete. 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.575+0000 7f7ee3980700 1 Processor -- start 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.575+0000 7f7ee3980700 1 -- start start 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee3980700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1282e0 0x7f7edc0944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee3980700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 0x7f7edc094a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee3980700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc130ec0 0x7f7edc094f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee3980700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0975d0 con 0x7f7edc1282e0 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee3980700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc09ac40 con 0x7f7edc130ec0 2024-04-25T11:02:38.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee3980700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0954b0 con 0x7f7edc12e590 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 0x7f7edc094a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 0x7f7edc094a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42240/0 (socket says 172.21.15.155:42240) 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/3608210641 learned_addr learned my addr 172.21.15.155:0/3608210641 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee1f1d700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc130ec0 0x7f7edc094f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc130ec0 msgr2=0x7f7edc094f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc130ec0 0x7f7edc094f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1282e0 msgr2=0x7f7edc0944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1282e0 0x7f7edc0944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.576+0000 7f7ee0f1b700 1 -- 172.21.15.155:0/3608210641 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed800d940 con 0x7f7edc12e590 2024-04-25T11:02:38.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.577+0000 7f7ee0f1b700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 0x7f7edc094a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800f8e0 tx=0x7f7ed8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.577+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80038a0 con 0x7f7edc12e590 2024-04-25T11:02:38.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.577+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8003a00 con 0x7f7edc12e590 2024-04-25T11:02:38.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.577+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc0965f0 con 0x7f7edc12e590 2024-04-25T11:02:38.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.577+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8003d70 con 0x7f7edc12e590 2024-04-25T11:02:38.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.578+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc0970b0 con 0x7f7edc12e590 2024-04-25T11:02:38.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.579+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc067020 con 0x7f7edc12e590 2024-04-25T11:02:38.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.585+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f7ed8016430 con 0x7f7edc12e590 2024-04-25T11:02:38.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.585+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f7ed8055a80 con 0x7f7edc12e590 2024-04-25T11:02:38.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.585+0000 7f7ee171c700 1 -- 172.21.15.155:0/3608210641 >> v1:172.21.15.155:6816/43298 conn(0x7f7ec00421a0 legacy=0x7f7ec0044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:38.587 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.586+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f7ed8020dc0 con 0x7f7edc12e590 2024-04-25T11:02:38.608 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.607+0000 7fceb9f71700 1 Processor -- start 2024-04-25T11:02:38.608 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.607+0000 7fceb9f71700 1 -- start start 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb9f71700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 0x7fceb406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb9f71700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fceb406f140 0x7fceb412f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb9f71700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb4072780 con 0x7fceb406ed20 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb9f71700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb406ca20 con 0x7fceb406f140 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb9f71700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceb406fb80 con 0x7fceb406e400 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 0x7fceb406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 0x7fceb406e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42256/0 (socket says 172.21.15.155:42256) 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 -- 172.21.15.155:0/3727332987 learned_addr learned my addr 172.21.15.155:0/3727332987 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:38.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb3fff700 1 --2- 172.21.15.155:0/3727332987 >> v2:172.21.15.155:3300/0 conn(0x7fceb406f140 0x7fceb412f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 -- 172.21.15.155:0/3727332987 >> v2:172.21.15.155:3300/0 conn(0x7fceb406f140 msgr2=0x7fceb412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:38.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 --2- 172.21.15.155:0/3727332987 >> v2:172.21.15.155:3300/0 conn(0x7fceb406f140 0x7fceb412f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 -- 172.21.15.155:0/3727332987 >> v1:172.21.15.155:6789/0 conn(0x7fceb406ed20 legacy=0x7fceb412ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:38.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.608+0000 7fceb37fe700 1 -- 172.21.15.155:0/3727332987 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb406c6e0 con 0x7fceb406e400 2024-04-25T11:02:38.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.609+0000 7fceb37fe700 1 --2- 172.21.15.155:0/3727332987 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 0x7fceb406e7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009920 tx=0x7fce9c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3dd840ed8a89f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.609+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a28062510 con 0x7f3a2806e400 2024-04-25T11:02:38.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.609+0000 7fceb27fc700 1 -- 172.21.15.155:0/3727332987 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004230 con 0x7fceb406e400 2024-04-25T11:02:38.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.609+0000 7fceb27fc700 1 -- 172.21.15.155:0/3727332987 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c004390 con 0x7fceb406e400 2024-04-25T11:02:38.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.609+0000 7fceb27fc700 1 -- 172.21.15.155:0/3727332987 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0036a0 con 0x7fceb406e400 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 -- 172.21.15.155:0/3727332987 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 msgr2=0x7fceb406e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 --2- 172.21.15.155:0/3727332987 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 0x7fceb406e7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009920 tx=0x7fce9c009c30 comp rx=0 tx=0).stop 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 -- 172.21.15.155:0/3727332987 shutdown_connections 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 --2- 172.21.15.155:0/3727332987 >> v2:172.21.15.155:3300/0 conn(0x7fceb406f140 0x7fceb412f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 --2- 172.21.15.155:0/3727332987 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406e400 0x7fceb406e7e0 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009920 tx=0x7fce9c009c30 comp rx=0 tx=0).stop 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 -- 172.21.15.155:0/3727332987 >> 172.21.15.155:0/3727332987 conn(0x7fceb409dac0 msgr2=0x7fceb409ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 -- 172.21.15.155:0/3727332987 shutdown_connections 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7fceb9f71700 1 -- 172.21.15.155:0/3727332987 wait complete. 2024-04-25T11:02:38.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.610+0000 7f3a0ffff700 1 -- 172.21.15.155:0/120016562 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a18039a10 con 0x7f3a2806e400 2024-04-25T11:02:38.613 INFO:teuthology.orchestra.run.smithi155.stdout:42949672981 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.611+0000 7fceb9f71700 1 Processor -- start 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 -- start start 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb406ed20 0x7fceb4134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406f140 0x7fceb4134a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb41363d0 0x7fceb4134f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb4096f70 con 0x7fceb406ed20 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb408b4c0 con 0x7fceb41363d0 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.612+0000 7fceb9f71700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceb408b1e0 con 0x7fceb406f140 2024-04-25T11:02:38.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb2ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406f140 0x7fceb4134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb2ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406f140 0x7fceb4134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42270/0 (socket says 172.21.15.155:42270) 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb2ffd700 1 -- 172.21.15.155:0/1128433976 learned_addr learned my addr 172.21.15.155:0/1128433976 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb3fff700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb41363d0 0x7fceb4134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb37fe700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb406ed20 0x7fceb4134510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb37fe700 1 -- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406f140 msgr2=0x7fceb4134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb37fe700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406f140 0x7fceb4134a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb37fe700 1 -- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb41363d0 msgr2=0x7fceb4134f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb37fe700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb41363d0 0x7fceb4134f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.613+0000 7fceb37fe700 1 -- 172.21.15.155:0/1128433976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c009550 con 0x7fceb406ed20 2024-04-25T11:02:38.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7fceb37fe700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb406ed20 0x7fceb4134510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fce9c015000 tx=0x7fce9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 >> v1:172.21.15.155:6816/43298 conn(0x7f3a140421a0 legacy=0x7f3a14044660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:38.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 msgr2=0x7f3a2812ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 0x7f3a2812ba30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c520 tx=0x7f3a1800c830 comp rx=0 tx=0).stop 2024-04-25T11:02:38.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 shutdown_connections 2024-04-25T11:02:38.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a281341e0 0x7f3a2812c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a2806e400 0x7f3a2812ba30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 --2- 172.21.15.155:0/120016562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3a2806eef0 0x7f3a2812bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.614+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 >> 172.21.15.155:0/120016562 conn(0x7f3a2809dac0 msgr2=0x7f3a280710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.615+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c003cf0 con 0x7fceb406ed20 2024-04-25T11:02:38.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.615+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c003e70 con 0x7fceb406ed20 2024-04-25T11:02:38.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.615+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c011850 con 0x7fceb406ed20 2024-04-25T11:02:38.618 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.615+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 shutdown_connections 2024-04-25T11:02:38.618 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.615+0000 7f3a2d8bf700 1 -- 172.21.15.155:0/120016562 wait complete. 2024-04-25T11:02:38.618 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.616+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb4099e90 con 0x7fceb406ed20 2024-04-25T11:02:38.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.616+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb41cddb0 con 0x7fceb406ed20 2024-04-25T11:02:38.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.617+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb4067020 con 0x7fceb406ed20 2024-04-25T11:02:38.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.617+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7fce9c021070 con 0x7fceb406ed20 2024-04-25T11:02:38.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.618+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fce9c056c90 con 0x7fceb406ed20 2024-04-25T11:02:38.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.618+0000 7fceb2ffd700 1 -- 172.21.15.155:0/1128433976 >> v1:172.21.15.155:6816/43298 conn(0x7fce940421d0 legacy=0x7fce94044690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:38.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.629+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7fce9c001530 con 0x7fceb406ed20 2024-04-25T11:02:38.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672983 got 42949672981 for osd.2 2024-04-25T11:02:38.657 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.656+0000 7f600d46c700 1 Processor -- start 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.656+0000 7f600d46c700 1 -- start start 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f600d46c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 0x7f6008093800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f600d46c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6008095510 0x7f600813aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f600d46c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600806ab70 con 0x7f6008094980 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f600d46c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6008096d30 con 0x7f6008095510 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f600d46c700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60080960a0 con 0x7f6008093420 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f6006ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 0x7f6008093800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f6006ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 0x7f6008093800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42280/0 (socket says 172.21.15.155:42280) 2024-04-25T11:02:38.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.657+0000 7f6006ffd700 1 -- 172.21.15.155:0/2962029932 learned_addr learned my addr 172.21.15.155:0/2962029932 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f60077fe700 1 --2- 172.21.15.155:0/2962029932 >> v2:172.21.15.155:3300/0 conn(0x7f6008095510 0x7f600813aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6005ffb700 1 -- 172.21.15.155:0/2962029932 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820525409 0 0) 0x7f600806ab70 con 0x7f6008094980 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6005ffb700 1 -- 172.21.15.155:0/2962029932 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff00035b0 con 0x7f6008094980 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6005ffb700 1 -- 172.21.15.155:0/2962029932 <== mon.0 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457883411 0 0) 0x7f5ff00035b0 con 0x7f6008094980 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f60077fe700 1 -- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 msgr2=0x7f6008093800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f60077fe700 1 --2- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 0x7f6008093800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f60077fe700 1 -- 172.21.15.155:0/2962029932 >> v1:172.21.15.155:6789/0 conn(0x7f6008094980 legacy=0x7f6008094e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f60077fe700 1 -- 172.21.15.155:0/2962029932 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008096f10 con 0x7f6008095510 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6006ffd700 1 --2- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 0x7f6008093800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f60077fe700 1 --2- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008095510 0x7f600813aed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6000009d90 tx=0x7f600000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2607fdd9953657ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6005ffb700 1 -- 172.21.15.155:0/2962029932 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000e070 con 0x7f6008095510 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6005ffb700 1 -- 172.21.15.155:0/2962029932 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6000003a20 con 0x7f6008095510 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.658+0000 7f6005ffb700 1 -- 172.21.15.155:0/2962029932 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc001670 con 0x7f6008095510 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.659+0000 7f600d46c700 1 -- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008095510 msgr2=0x7f600813aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.659+0000 7f600d46c700 1 --2- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008095510 0x7f600813aed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6000009d90 tx=0x7f600000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.659+0000 7f600d46c700 1 -- 172.21.15.155:0/2962029932 shutdown_connections 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.659+0000 7f600d46c700 1 --2- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008095510 0x7f600813aed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.659+0000 7f600d46c700 1 --2- 172.21.15.155:0/2962029932 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008093420 0x7f6008093800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.659+0000 7f600d46c700 1 -- 172.21.15.155:0/2962029932 >> 172.21.15.155:0/2962029932 conn(0x7f600809d480 msgr2=0x7f600809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.660+0000 7f600d46c700 1 -- 172.21.15.155:0/2962029932 shutdown_connections 2024-04-25T11:02:38.662 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.660+0000 7f600d46c700 1 -- 172.21.15.155:0/2962029932 wait complete. 2024-04-25T11:02:38.663 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.661+0000 7f600d46c700 1 Processor -- start 2024-04-25T11:02:38.663 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.661+0000 7f600d46c700 1 -- start start 2024-04-25T11:02:38.663 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f600d46c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008093420 0x7f6008139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.663 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f600d46c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008094980 0x7f60081395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f600d46c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6008095510 0x7f6008139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f600d46c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008133700 con 0x7f6008093420 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f600d46c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008091ed0 con 0x7f6008095510 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f600d46c700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6008072300 con 0x7f6008094980 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60067fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008094980 0x7f60081395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60067fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008094980 0x7f60081395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42294/0 (socket says 172.21.15.155:42294) 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60067fc700 1 -- 172.21.15.155:0/3018274829 learned_addr learned my addr 172.21.15.155:0/3018274829 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60077fe700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6008095510 0x7f6008139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60077fe700 1 -- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008094980 msgr2=0x7f60081395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60077fe700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008094980 0x7f60081395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60077fe700 1 -- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008093420 msgr2=0x7f6008139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60077fe700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008093420 0x7f6008139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.662+0000 7f60077fe700 1 -- 172.21.15.155:0/3018274829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8009550 con 0x7f6008095510 2024-04-25T11:02:38.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.663+0000 7f60077fe700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6008095510 0x7f6008139ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6008095960 tx=0x7f6000004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:38.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.663+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000004440 con 0x7f6008095510 2024-04-25T11:02:38.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.663+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6000009a70 con 0x7f6008095510 2024-04-25T11:02:38.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.663+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60000045a0 con 0x7f6008095510 2024-04-25T11:02:38.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.663+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000004910 con 0x7f6008095510 2024-04-25T11:02:38.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.663+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008092800 con 0x7f6008095510 2024-04-25T11:02:38.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.665+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f6000012430 con 0x7f6008095510 2024-04-25T11:02:38.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.665+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f60000577c0 con 0x7f6008095510 2024-04-25T11:02:38.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.665+0000 7f6006ffd700 1 -- 172.21.15.155:0/3018274829 >> v1:172.21.15.155:6816/43298 conn(0x7f5ff40421d0 legacy=0x7f5ff4044690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:38.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.665+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008062510 con 0x7f6008095510 2024-04-25T11:02:38.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.672+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f6000053c20 con 0x7f6008095510 2024-04-25T11:02:38.707 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:38.706+0000 7f94bd147200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:02:38.711 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:38.711+0000 7f5ff79b3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:02:38.786 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.786+0000 7f7ee171c700 1 -- 172.21.15.155:0/3608210641 >> v1:172.21.15.155:6816/43298 conn(0x7f7ec00421a0 legacy=0x7f7ec0044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:38.791 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.791+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fceb4062510 con 0x7fceb406ed20 2024-04-25T11:02:38.791 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.791+0000 7fceabfff700 1 -- 172.21.15.155:0/1128433976 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce9c029030 con 0x7fceb406ed20 2024-04-25T11:02:38.792 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:38.792 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.791+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7edc12daf0 con 0x7f7edc12e590 2024-04-25T11:02:38.792 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.792+0000 7f7ed1ffb700 1 -- 172.21.15.155:0/3608210641 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ed801b020 con 0x7f7edc12e590 2024-04-25T11:02:38.793 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:38.793 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.793+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 >> v1:172.21.15.155:6816/43298 conn(0x7fce940421d0 legacy=0x7fce94044690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:38.794 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.793+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb406ed20 msgr2=0x7fceb4134510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.794 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.793+0000 7fceb9f71700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb406ed20 0x7fceb4134510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fce9c015000 tx=0x7fce9c004750 comp rx=0 tx=0).stop 2024-04-25T11:02:38.794 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 shutdown_connections 2024-04-25T11:02:38.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb41363d0 0x7fceb4134f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb406ed20 0x7fceb4134510 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 --2- 172.21.15.155:0/1128433976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fceb406f140 0x7fceb4134a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 >> 172.21.15.155:0/1128433976 conn(0x7fceb409dac0 msgr2=0x7fceb4071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 shutdown_connections 2024-04-25T11:02:38.795 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.794+0000 7fceb9f71700 1 -- 172.21.15.155:0/1128433976 wait complete. 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 >> v1:172.21.15.155:6816/43298 conn(0x7f7ec00421a0 legacy=0x7f7ec0044660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 msgr2=0x7f7edc094a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 0x7f7edc094a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800f8e0 tx=0x7f7ed8004750 comp rx=0 tx=0).stop 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 shutdown_connections 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc130ec0 0x7f7edc094f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1282e0 0x7f7edc0944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 --2- 172.21.15.155:0/3608210641 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7edc12e590 0x7f7edc094a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 >> 172.21.15.155:0/3608210641 conn(0x7f7edc09dac0 msgr2=0x7f7edc071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 shutdown_connections 2024-04-25T11:02:38.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.795+0000 7f7ee3980700 1 -- 172.21.15.155:0/3608210641 wait complete. 2024-04-25T11:02:38.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.0 2024-04-25T11:02:38.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.3 2024-04-25T11:02:38.866 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.866+0000 7f6006ffd700 1 -- 172.21.15.155:0/3018274829 >> v1:172.21.15.155:6816/43298 conn(0x7f5ff40421d0 legacy=0x7f5ff4044690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:38.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.888+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6008122310 con 0x7f6008095510 2024-04-25T11:02:38.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.888+0000 7f5fef7fe700 1 -- 172.21.15.155:0/3018274829 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6000034030 con 0x7f6008095510 2024-04-25T11:02:38.889 INFO:teuthology.orchestra.run.smithi155.stdout:42949672980 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 >> v1:172.21.15.155:6816/43298 conn(0x7f5ff40421d0 legacy=0x7f5ff4044690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6008095510 msgr2=0x7f6008139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6008095510 0x7f6008139ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6008095960 tx=0x7f6000004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 shutdown_connections 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6008095510 0x7f6008139ae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008093420 0x7f6008139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 --2- 172.21.15.155:0/3018274829 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6008094980 0x7f60081395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:38.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 >> 172.21.15.155:0/3018274829 conn(0x7f600809d480 msgr2=0x7f600806fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:38.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 shutdown_connections 2024-04-25T11:02:38.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:38.890+0000 7f600d46c700 1 -- 172.21.15.155:0/3018274829 wait complete. 2024-04-25T11:02:38.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672982 got 42949672980 for osd.1 2024-04-25T11:02:38.988 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:38.987+0000 7f94bd147200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:02:39.013 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:39.012+0000 7f5ff79b3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:02:39.379 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:39.378+0000 7f94bd147200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:02:39.382 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:39.382+0000 7f5ff79b3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:02:39.518 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:39.518+0000 7f94bd147200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:02:39.519 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:39.519+0000 7f5ff79b3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:02:39.633 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:02:39.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.765+0000 7f0c2bb79700 1 Processor -- start 2024-04-25T11:02:39.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2bb79700 1 -- start start 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2bb79700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c2406e3a0 0x7f0c24133840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2bb79700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c24133d80 0x7f0c24136170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2bb79700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2406faa0 con 0x7f0c2406d8b0 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2bb79700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2406ebc0 con 0x7f0c2406e3a0 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2bb79700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240721b0 con 0x7f0c24133d80 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c29114700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c2406e3a0 0x7f0c24133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c29915700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0c2406d8b0 0x7f0c2406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47172/0 (socket says 172.21.15.155:47172) 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c29915700 1 -- 172.21.15.155:0/2274226868 learned_addr learned my addr 172.21.15.155:0/2274226868 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.766+0000 7f0c2a116700 1 --2- 172.21.15.155:0/2274226868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c24133d80 0x7f0c24136170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c2a116700 1 -- 172.21.15.155:0/2274226868 >> v2:172.21.15.155:3300/0 conn(0x7f0c2406e3a0 msgr2=0x7f0c24133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c2a116700 1 --2- 172.21.15.155:0/2274226868 >> v2:172.21.15.155:3300/0 conn(0x7f0c2406e3a0 0x7f0c24133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c2a116700 1 -- 172.21.15.155:0/2274226868 >> v1:172.21.15.155:6789/0 conn(0x7f0c2406d8b0 legacy=0x7f0c2406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c28913700 1 -- 172.21.15.155:0/2274226868 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910873304 0 0) 0x7f0c2406faa0 con 0x7f0c2406d8b0 2024-04-25T11:02:39.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c2a116700 1 -- 172.21.15.155:0/2274226868 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2406f660 con 0x7f0c24133d80 2024-04-25T11:02:39.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c29114700 1 --2- 172.21.15.155:0/2274226868 >> v2:172.21.15.155:3300/0 conn(0x7f0c2406e3a0 0x7f0c24133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:39.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c2a116700 1 --2- 172.21.15.155:0/2274226868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c24133d80 0x7f0c24136170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009dc0 tx=0x7f0c1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bae9e679b8ac92d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:39.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c28913700 1 -- 172.21.15.155:0/2274226868 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00e070 con 0x7f0c24133d80 2024-04-25T11:02:39.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c28913700 1 -- 172.21.15.155:0/2274226868 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c00bae0 con 0x7f0c24133d80 2024-04-25T11:02:39.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.767+0000 7f0c28913700 1 -- 172.21.15.155:0/2274226868 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00be90 con 0x7f0c24133d80 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2274226868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c24133d80 msgr2=0x7f0c24136170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2274226868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c24133d80 0x7f0c24136170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009dc0 tx=0x7f0c1c00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2274226868 shutdown_connections 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2274226868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c24133d80 0x7f0c24136170 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2274226868 >> v2:172.21.15.155:3300/0 conn(0x7f0c2406e3a0 0x7f0c24133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2274226868 >> 172.21.15.155:0/2274226868 conn(0x7f0c2409ecb0 msgr2=0x7f0c2409f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2274226868 shutdown_connections 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.768+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2274226868 wait complete. 2024-04-25T11:02:39.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 Processor -- start 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 -- start start 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0c2406d8b0 0x7f0c2412fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 0x7f0c24130040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c24133d80 0x7f0c241c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2409b290 con 0x7f0c24133d80 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24132010 con 0x7f0c2406e3a0 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.769+0000 7f0c2bb79700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2409a9e0 con 0x7f0c2406d8b0 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 0x7f0c24130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c2a116700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c24133d80 0x7f0c241c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 0x7f0c24130040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48498/0 (socket says 172.21.15.155:48498) 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 -- 172.21.15.155:0/2693263855 learned_addr learned my addr 172.21.15.155:0/2693263855 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29915700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0c2406d8b0 0x7f0c2412fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 -- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0c2406d8b0 msgr2=0x7f0c2412fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0c2406d8b0 0x7f0c2412fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 -- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c24133d80 msgr2=0x7f0c241c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c24133d80 0x7f0c241c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 -- 172.21.15.155:0/2693263855 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140094a0 con 0x7f0c2406e3a0 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c2a116700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c24133d80 0x7f0c241c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c29114700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 0x7f0c24130040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400ec10 tx=0x7f0c1400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400c850 con 0x7f0c2406e3a0 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1400c9b0 con 0x7f0c2406e3a0 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c009a70 con 0x7f0c2406e3a0 2024-04-25T11:02:39.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400cd20 con 0x7f0c2406e3a0 2024-04-25T11:02:39.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.770+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c24130580 con 0x7f0c2406e3a0 2024-04-25T11:02:39.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.771+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c24062510 con 0x7f0c2406e3a0 2024-04-25T11:02:39.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.771+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 48664+0+0 (secure 0 0 0) 0x7f0c14013470 con 0x7f0c2406e3a0 2024-04-25T11:02:39.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.772+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f0c140559b0 con 0x7f0c2406e3a0 2024-04-25T11:02:39.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.772+0000 7f0c29915700 1 -- 172.21.15.155:0/2693263855 >> v1:172.21.15.155:6816/43298 conn(0x7f0c080421a0 legacy=0x7f0c08044660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.155:6816/43298 2024-04-25T11:02:39.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.775+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f0c14036240 con 0x7f0c2406e3a0 2024-04-25T11:02:39.811 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:02:39.814 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:02:39.839 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:39.839+0000 7f94bd147200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:02:39.846 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:39.846+0000 7f5ff79b3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:02:39.905 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:02:39.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.956+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0c2406b910 con 0x7f0c2406e3a0 2024-04-25T11:02:39.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.956+0000 7f0c1a7fc700 1 -- 172.21.15.155:0/2693263855 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c14017070 con 0x7f0c2406e3a0 2024-04-25T11:02:39.963 INFO:teuthology.orchestra.run.smithi155.stdout:42949672981 2024-04-25T11:02:39.964 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T11:02:39.963+0000 7f94bd147200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:02:39.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.964+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 >> v1:172.21.15.155:6816/43298 conn(0x7f0c080421a0 legacy=0x7f0c08044660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:39.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.964+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 msgr2=0x7f0c24130040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.964+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 0x7f0c24130040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400ec10 tx=0x7f0c1400ef20 comp rx=0 tx=0).stop 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 shutdown_connections 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c24133d80 0x7f0c241c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0c2406d8b0 0x7f0c2412fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 --2- 172.21.15.155:0/2693263855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2406e3a0 0x7f0c24130040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 >> 172.21.15.155:0/2693263855 conn(0x7f0c2409ecb0 msgr2=0x7f0c2409b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 shutdown_connections 2024-04-25T11:02:39.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.965+0000 7f0c2bb79700 1 -- 172.21.15.155:0/2693263855 wait complete. 2024-04-25T11:02:39.969 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:39.968+0000 7f5ff79b3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:02:39.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.981+0000 7f14c7807700 1 Processor -- start 2024-04-25T11:02:39.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.981+0000 7f14c7807700 1 -- start start 2024-04-25T11:02:39.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672983 got 42949672981 for osd.2 2024-04-25T11:02:39.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.982+0000 7f14c7807700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14c012f9d0 0x7f14c012fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.982+0000 7f14c7807700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0132a50 0x7f14c0132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.982+0000 7f14c7807700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0133410 0x7f14c0139d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.982+0000 7f14c7807700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c006fb70 con 0x7f14c012f9d0 2024-04-25T11:02:39.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.982+0000 7f14c7807700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00726a0 con 0x7f14c0133410 2024-04-25T11:02:39.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.982+0000 7f14c7807700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14c012f5f0 con 0x7f14c0132a50 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.983+0000 7f14c5da4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0133410 0x7f14c0139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.983+0000 7f14c5da4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0133410 0x7f14c0139d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48520/0 (socket says 172.21.15.155:48520) 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.983+0000 7f14c5da4700 1 -- 172.21.15.155:0/2650211299 learned_addr learned my addr 172.21.15.155:0/2650211299 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 --2- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0132a50 0x7f14c0132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c55a3700 1 --2- 172.21.15.155:0/2650211299 >> v2:172.21.15.155:3300/0 conn(0x7f14c012f9d0 0x7f14c012fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 -- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0133410 msgr2=0x7f14c0139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 --2- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0133410 0x7f14c0139d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 -- 172.21.15.155:0/2650211299 >> v2:172.21.15.155:3300/0 conn(0x7f14c012f9d0 msgr2=0x7f14c012fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:39.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 --2- 172.21.15.155:0/2650211299 >> v2:172.21.15.155:3300/0 conn(0x7f14c012f9d0 0x7f14c012fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 -- 172.21.15.155:0/2650211299 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c01338a0 con 0x7f14c0132a50 2024-04-25T11:02:39.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14c4da2700 1 --2- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0132a50 0x7f14c0132ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009870 tx=0x7f14b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c709ef4d1e56efe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:39.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14b7fff700 1 -- 172.21.15.155:0/2650211299 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011830 con 0x7f14c0132a50 2024-04-25T11:02:39.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.984+0000 7f14b7fff700 1 -- 172.21.15.155:0/2650211299 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0011990 con 0x7f14c0132a50 2024-04-25T11:02:39.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.985+0000 7f14b7fff700 1 -- 172.21.15.155:0/2650211299 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011d00 con 0x7f14c0132a50 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 -- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0132a50 msgr2=0x7f14c0132ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 --2- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0132a50 0x7f14c0132ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009870 tx=0x7f14b0009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 -- 172.21.15.155:0/2650211299 shutdown_connections 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 --2- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0133410 0x7f14c0139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 --2- 172.21.15.155:0/2650211299 >> v2:172.21.15.155:3300/0 conn(0x7f14c012f9d0 0x7f14c012fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 --2- 172.21.15.155:0/2650211299 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0132a50 0x7f14c0132ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 -- 172.21.15.155:0/2650211299 >> 172.21.15.155:0/2650211299 conn(0x7f14c009c8d0 msgr2=0x7f14c009cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.986+0000 7f14c7807700 1 -- 172.21.15.155:0/2650211299 shutdown_connections 2024-04-25T11:02:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.987+0000 7f14c7807700 1 -- 172.21.15.155:0/2650211299 wait complete. 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 Processor -- start 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 -- start start 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 0x7f14c0129960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 0x7f14c0121990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0133410 0x7f14c0121ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c0124b40 con 0x7f14c012f9d0 2024-04-25T11:02:39.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c0129fd0 con 0x7f14c0132a50 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.988+0000 7f14c7807700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14c0124360 con 0x7f14c0133410 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c55a3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 0x7f14c0129960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 0x7f14c0121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c55a3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 0x7f14c0129960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51978/0 (socket says 172.21.15.155:51978) 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 0x7f14c0121990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48530/0 (socket says 172.21.15.155:48530) 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c55a3700 1 -- 172.21.15.155:0/331043191 learned_addr learned my addr 172.21.15.155:0/331043191 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c5da4700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0133410 0x7f14c0121ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 -- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0133410 msgr2=0x7f14c0121ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0133410 0x7f14c0121ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 -- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 msgr2=0x7f14c0129960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 0x7f14c0129960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 -- 172.21.15.155:0/331043191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00094a0 con 0x7f14c0132a50 2024-04-25T11:02:39.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c55a3700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 0x7f14c0129960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c4da2700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 0x7f14c0121990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14b0016000 tx=0x7f14b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0004450 con 0x7f14c0132a50 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c0124640 con 0x7f14c0132a50 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0041070 con 0x7f14c0132a50 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.989+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b00038e0 con 0x7f14c0132a50 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.990+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c0122410 con 0x7f14c0132a50 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.990+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 6) v1 ==== 51728+0+0 (secure 0 0 0) 0x7f14b0003a40 con 0x7f14c0132a50 2024-04-25T11:02:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.991+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f14b0043070 con 0x7f14c0132a50 2024-04-25T11:02:39.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.991+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c0062510 con 0x7f14c0132a50 2024-04-25T11:02:39.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:39.995+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f14b00086b0 con 0x7f14c0132a50 2024-04-25T11:02:40.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.022+0000 7f58610d8700 1 Processor -- start 2024-04-25T11:02:40.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.022+0000 7f58610d8700 1 -- start start 2024-04-25T11:02:40.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f58610d8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c12b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f58610d8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c12bc70 0x7f585c0961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f58610d8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c066870 con 0x7f585c096710 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f58610d8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c09a010 con 0x7f585c12bc70 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f58610d8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f585c072780 con 0x7f585c12b350 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585a59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c12bc70 0x7f585c0961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585b59e700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f585c096710 0x7f585c132900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47176/0 (socket says 172.21.15.155:47176) 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585b59e700 1 -- 172.21.15.155:0/1468782107 learned_addr learned my addr 172.21.15.155:0/1468782107 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585a59c700 1 -- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 msgr2=0x7f585c12b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585a59c700 1 --2- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c12b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585a59c700 1 -- 172.21.15.155:0/1468782107 >> v1:172.21.15.155:6789/0 conn(0x7f585c096710 legacy=0x7f585c132900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585a59c700 1 -- 172.21.15.155:0/1468782107 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c071080 con 0x7f585c12bc70 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f585a59c700 1 --2- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c12bc70 0x7f585c0961d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f585000bb10 tx=0x7f585000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac3a09317b7dbaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f5859d9b700 1 -- 172.21.15.155:0/1468782107 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585000dd30 con 0x7f585c12bc70 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f5859d9b700 1 -- 172.21.15.155:0/1468782107 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585000de90 con 0x7f585c12bc70 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.023+0000 7f5859d9b700 1 -- 172.21.15.155:0/1468782107 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850010690 con 0x7f585c12bc70 2024-04-25T11:02:40.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.024+0000 7f58610d8700 1 -- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c12bc70 msgr2=0x7f585c0961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.024+0000 7f58610d8700 1 --2- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c12bc70 0x7f585c0961d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f585000bb10 tx=0x7f585000be20 comp rx=0 tx=0).stop 2024-04-25T11:02:40.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.024+0000 7f58610d8700 1 -- 172.21.15.155:0/1468782107 shutdown_connections 2024-04-25T11:02:40.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.024+0000 7f58610d8700 1 --2- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c12b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.024+0000 7f58610d8700 1 --2- 172.21.15.155:0/1468782107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c12bc70 0x7f585c0961d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.024+0000 7f58610d8700 1 -- 172.21.15.155:0/1468782107 >> 172.21.15.155:0/1468782107 conn(0x7f585c09d1c0 msgr2=0x7f585c099690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:40.026 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.025+0000 7f58610d8700 1 -- 172.21.15.155:0/1468782107 shutdown_connections 2024-04-25T11:02:40.027 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.026+0000 7f58610d8700 1 -- 172.21.15.155:0/1468782107 wait complete. 2024-04-25T11:02:40.027 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 Processor -- start 2024-04-25T11:02:40.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 -- start start 2024-04-25T11:02:40.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 0x7f585c124f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c125490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c12bc70 0x7f585c1c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c127100 con 0x7f585c12bc70 2024-04-25T11:02:40.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c126040 con 0x7f585c096710 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.027+0000 7f58610d8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f585c12a860 con 0x7f585c12b350 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 0x7f585c124f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 0x7f585c124f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48610/0 (socket says 172.21.15.155:48610) 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 -- 172.21.15.155:0/3469389248 learned_addr learned my addr 172.21.15.155:0/3469389248 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585a59c700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c125490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585b59e700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c12bc70 0x7f585c1c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 -- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 msgr2=0x7f585c125490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c125490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 -- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c12bc70 msgr2=0x7f585c1c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c12bc70 0x7f585c1c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 -- 172.21.15.155:0/3469389248 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585000b740 con 0x7f585c096710 2024-04-25T11:02:40.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.028+0000 7f585ad9d700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 0x7f585c124f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b560 tx=0x7f584c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:40.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.035+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c011830 con 0x7f585c096710 2024-04-25T11:02:40.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.035+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c011990 con 0x7f585c096710 2024-04-25T11:02:40.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.035+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c011d40 con 0x7f585c096710 2024-04-25T11:02:40.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.036+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585c128360 con 0x7f585c096710 2024-04-25T11:02:40.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.036+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585c1259d0 con 0x7f585c096710 2024-04-25T11:02:40.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.037+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585c062510 con 0x7f585c096710 2024-04-25T11:02:40.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.038+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 6) v1 ==== 51728+0+0 (secure 0 0 0) 0x7f584c01a400 con 0x7f585c096710 2024-04-25T11:02:40.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.038+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f584c059250 con 0x7f585c096710 2024-04-25T11:02:40.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.041+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7f584c0370d0 con 0x7f585c096710 2024-04-25T11:02:40.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.095+0000 7fc3a532c700 1 Processor -- start 2024-04-25T11:02:40.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.095+0000 7fc3a532c700 1 -- start start 2024-04-25T11:02:40.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc3a532c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3a012e770 0x7fc3a012ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc3a532c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012f1c0 0x7fc3a012b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc3a532c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a012f640 con 0x7fc3a012b770 2024-04-25T11:02:40.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc3a532c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a006fc40 con 0x7fc3a012e770 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc3a532c700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a006f2b0 con 0x7fc3a012f1c0 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39f7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc3a012b770 0x7fc3a0135b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47190/0 (socket says 172.21.15.155:47190) 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39effd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3a012e770 0x7fc3a012ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39f7fe700 1 -- 172.21.15.155:0/2475179100 learned_addr learned my addr 172.21.15.155:0/2475179100 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39e7fc700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012f1c0 0x7fc3a012b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39effd700 1 -- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012f1c0 msgr2=0x7fc3a012b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39effd700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012f1c0 0x7fc3a012b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39effd700 1 -- 172.21.15.155:0/2475179100 >> v1:172.21.15.155:6789/0 conn(0x7fc3a012b770 legacy=0x7fc3a0135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39dffb700 1 -- 172.21.15.155:0/2475179100 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594324511 0 0) 0x7fc3a012f640 con 0x7fc3a012b770 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39effd700 1 -- 172.21.15.155:0/2475179100 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a006f420 con 0x7fc3a012e770 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39e7fc700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012f1c0 0x7fc3a012b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.096+0000 7fc39effd700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012e770 0x7fc3a012ec80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc390009840 tx=0x7fc390009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13555aa744104067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc39dffb700 1 -- 172.21.15.155:0/2475179100 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390011830 con 0x7fc3a012e770 2024-04-25T11:02:40.098 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc39dffb700 1 -- 172.21.15.155:0/2475179100 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc390011990 con 0x7fc3a012e770 2024-04-25T11:02:40.099 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc39dffb700 1 -- 172.21.15.155:0/2475179100 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390011d20 con 0x7fc3a012e770 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc3a532c700 1 -- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012e770 msgr2=0x7fc3a012ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012e770 0x7fc3a012ec80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc390009840 tx=0x7fc390009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc3a532c700 1 -- 172.21.15.155:0/2475179100 shutdown_connections 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012e770 0x7fc3a012ec80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2475179100 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012f1c0 0x7fc3a012b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.097+0000 7fc3a532c700 1 -- 172.21.15.155:0/2475179100 >> 172.21.15.155:0/2475179100 conn(0x7fc3a00a0c70 msgr2=0x7fc3a00a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.098+0000 7fc3a532c700 1 -- 172.21.15.155:0/2475179100 shutdown_connections 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.098+0000 7fc3a532c700 1 -- 172.21.15.155:0/2475179100 wait complete. 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.099+0000 7fc3a532c700 1 Processor -- start 2024-04-25T11:02:40.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.099+0000 7fc3a532c700 1 -- start start 2024-04-25T11:02:40.102 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 0x7fc3a0125f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012e770 0x7fc3a01264d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a012f1c0 0x7fc3a0126a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a008f720 con 0x7fc3a012b770 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a01284c0 con 0x7fc3a012f1c0 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a01280f0 con 0x7fc3a012e770 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 0x7fc3a0125f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 0x7fc3a0125f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:52054/0 (socket says 172.21.15.155:52054) 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39effd700 1 -- 172.21.15.155:0/2221043176 learned_addr learned my addr 172.21.15.155:0/2221043176 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012e770 0x7fc3a01264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 -- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a012f1c0 msgr2=0x7fc3a0126a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a012f1c0 0x7fc3a0126a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 -- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 msgr2=0x7fc3a0125f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 0x7fc3a0125f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 -- 172.21.15.155:0/2221043176 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940094a0 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39effd700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 0x7fc3a0125f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc39e7fc700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012e770 0x7fc3a01264d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc39400e9b0 tx=0x7fc39400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400c850 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39400c9b0 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.100+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3900094a0 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.101+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400cd60 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.101+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a0128c20 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.101+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 6) v1 ==== 51728+0+0 (secure 0 0 0) 0x7fc394013470 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.102+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a0062510 con 0x7fc3a012e770 2024-04-25T11:02:40.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.102+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7fc394058d50 con 0x7fc3a012e770 2024-04-25T11:02:40.110 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.109+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+130055 (secure 0 0 0) 0x7fc394017080 con 0x7fc3a012e770 2024-04-25T11:02:40.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.200+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f585c128580 con 0x7f585c096710 2024-04-25T11:02:40.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.200+0000 7f58437fe700 1 -- 172.21.15.155:0/3469389248 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f584c03a8c0 con 0x7f585c096710 2024-04-25T11:02:40.203 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 msgr2=0x7f585c124f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 0x7f585c124f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b560 tx=0x7f584c00b870 comp rx=0 tx=0).stop 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 shutdown_connections 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c12bc70 0x7f585c1c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c096710 0x7f585c124f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 --2- 172.21.15.155:0/3469389248 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f585c12b350 0x7f585c125490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 >> 172.21.15.155:0/3469389248 conn(0x7f585c09d1c0 msgr2=0x7f585c097590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 shutdown_connections 2024-04-25T11:02:40.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.202+0000 7f58610d8700 1 -- 172.21.15.155:0/3469389248 wait complete. 2024-04-25T11:02:40.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.210+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f14c0123b80 con 0x7f14c0132a50 2024-04-25T11:02:40.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.210+0000 7f14b5ffb700 1 -- 172.21.15.155:0/331043191 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14b004d020 con 0x7f14c0132a50 2024-04-25T11:02:40.211 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 msgr2=0x7f14c0121990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 0x7f14c0121990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14b0016000 tx=0x7f14b0004750 comp rx=0 tx=0).stop 2024-04-25T11:02:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 shutdown_connections 2024-04-25T11:02:40.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f14c0133410 0x7f14c0121ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c012f9d0 0x7f14c0129960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 --2- 172.21.15.155:0/331043191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c0132a50 0x7f14c0121990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 >> 172.21.15.155:0/331043191 conn(0x7f14c009c8d0 msgr2=0x7f14c012a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:40.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 shutdown_connections 2024-04-25T11:02:40.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.212+0000 7f14c7807700 1 -- 172.21.15.155:0/331043191 wait complete. 2024-04-25T11:02:40.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.3 2024-04-25T11:02:40.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.0 2024-04-25T11:02:40.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.276+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc3a0067930 con 0x7fc3a012e770 2024-04-25T11:02:40.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.277+0000 7fc3877fe700 1 -- 172.21.15.155:0/2221043176 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc39403f090 con 0x7fc3a012e770 2024-04-25T11:02:40.277 INFO:teuthology.orchestra.run.smithi155.stdout:42949672980 2024-04-25T11:02:40.279 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012e770 msgr2=0x7fc3a01264d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:40.279 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012e770 0x7fc3a01264d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc39400e9b0 tx=0x7fc39400ecc0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 shutdown_connections 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a012f1c0 0x7fc3a0126a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a012b770 0x7fc3a0125f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 --2- 172.21.15.155:0/2221043176 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3a012e770 0x7fc3a01264d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.278+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 >> 172.21.15.155:0/2221043176 conn(0x7fc3a00a0c70 msgr2=0x7fc3a0070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.279+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 shutdown_connections 2024-04-25T11:02:40.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:40.279+0000 7fc3a532c700 1 -- 172.21.15.155:0/2221043176 wait complete. 2024-04-25T11:02:40.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672982 got 42949672980 for osd.1 2024-04-25T11:02:40.981 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:40.980+0000 7f5fd2f84700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-25T11:02:40.982 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:40.981+0000 7f5fd2f84700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T11:02:40.982 INFO:tasks.ceph.mgr.y.smithi155.stderr:2024-04-25T11:02:40.981+0000 7f5fd2f84700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T11:02:40.984 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:02:41.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.111+0000 7f37a4326700 1 Processor -- start 2024-04-25T11:02:41.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.111+0000 7f37a4326700 1 -- start start 2024-04-25T11:02:41.114 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a4326700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f379c06e400 0x7f379c06e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a4326700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 0x7f379c12ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a4326700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 0x7f379c12efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a4326700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c06ca20 con 0x7f379c06e400 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a4326700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c072780 con 0x7f379c06f140 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a4326700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f379c06fb80 con 0x7f379c06ed20 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a28c3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 0x7f379c12efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a18c1700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 0x7f379c12ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a18c1700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 0x7f379c12ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:42410/0 (socket says 172.21.15.155:42410) 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a28c3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 0x7f379c12efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48636/0 (socket says 172.21.15.155:48636) 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a18c1700 1 -- 172.21.15.155:0/1618771097 learned_addr learned my addr 172.21.15.155:0/1618771097 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 --2- 172.21.15.155:0/1618771097 >> v2:172.21.15.155:3300/0 conn(0x7f379c06e400 0x7f379c06e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 -- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 msgr2=0x7f379c12ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 0x7f379c12ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 -- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 msgr2=0x7f379c12efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 0x7f379c12efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 -- 172.21.15.155:0/1618771097 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c06c6e0 con 0x7f379c06e400 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a28c3700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 0x7f379c12efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a18c1700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 0x7f379c12ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a20c2700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06e400 0x7f379c06e7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f378c009840 tx=0x7f378c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebccbcefede13162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a10c0700 1 -- 172.21.15.155:0/1618771097 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c011830 con 0x7f379c06e400 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a10c0700 1 -- 172.21.15.155:0/1618771097 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c011990 con 0x7f379c06e400 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.112+0000 7f37a10c0700 1 -- 172.21.15.155:0/1618771097 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c011d20 con 0x7f379c06e400 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 -- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06e400 msgr2=0x7f379c06e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06e400 0x7f379c06e7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f378c009840 tx=0x7f378c009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:41.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 -- 172.21.15.155:0/1618771097 shutdown_connections 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06f140 0x7f379c12efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06e400 0x7f379c06e7e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 --2- 172.21.15.155:0/1618771097 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06ed20 0x7f379c12ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 -- 172.21.15.155:0/1618771097 >> 172.21.15.155:0/1618771097 conn(0x7f379c09dac0 msgr2=0x7f379c09ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 -- 172.21.15.155:0/1618771097 shutdown_connections 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.113+0000 7f37a4326700 1 -- 172.21.15.155:0/1618771097 wait complete. 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.114+0000 7f37a4326700 1 Processor -- start 2024-04-25T11:02:41.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.114+0000 7f37a4326700 1 -- start start 2024-04-25T11:02:41.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 0x7f379c095e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06ed20 0x7f379c0963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06f140 0x7f379c0968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c093f40 con 0x7f379c06ed20 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c095ba0 con 0x7f379c06e400 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0952d0 con 0x7f379c06f140 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 0x7f379c095e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 0x7f379c095e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:48650/0 (socket says 172.21.15.155:48650) 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 -- 172.21.15.155:0/3483188315 learned_addr learned my addr 172.21.15.155:0/3483188315 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a28c3700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06f140 0x7f379c0968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a18c1700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06ed20 0x7f379c0963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 -- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06f140 msgr2=0x7f379c0968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06f140 0x7f379c0968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 -- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06ed20 msgr2=0x7f379c0963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06ed20 0x7f379c0963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 -- 172.21.15.155:0/3483188315 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798009550 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a20c2700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 0x7f379c095e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f378c009fd0 tx=0x7f378c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c004240 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c01f070 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c0045e0 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3794009a70 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.115+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c0094a0 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.116+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f378c021030 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.117+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f378c05d480 con 0x7f379c06e400 2024-04-25T11:02:41.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.117+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c067020 con 0x7f379c06e400 2024-04-25T11:02:41.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.121+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f378c03eda0 con 0x7f379c06e400 2024-04-25T11:02:41.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:02:41.225 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:02:41.291 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:02:41.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.291+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f379c1348f0 con 0x7f379c06e400 2024-04-25T11:02:41.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.292+0000 7f3792ffd700 1 -- 172.21.15.155:0/3483188315 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f378c03e9b0 con 0x7f379c06e400 2024-04-25T11:02:41.293 INFO:teuthology.orchestra.run.smithi155.stdout:42949672981 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 >> v1:172.21.15.155:6816/49624 conn(0x7f3780047440 legacy=0x7f3780049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 msgr2=0x7f379c095e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 0x7f379c095e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f378c009fd0 tx=0x7f378c004750 comp rx=0 tx=0).stop 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 shutdown_connections 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f379c06f140 0x7f379c0968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f379c06e400 0x7f379c095e60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 --2- 172.21.15.155:0/3483188315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f379c06ed20 0x7f379c0963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 >> 172.21.15.155:0/3483188315 conn(0x7f379c09dac0 msgr2=0x7f379c131a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 shutdown_connections 2024-04-25T11:02:41.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.294+0000 7f37a4326700 1 -- 172.21.15.155:0/3483188315 wait complete. 2024-04-25T11:02:41.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672983 got 42949672981 for osd.2 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 Processor -- start 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 -- start start 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c093de0 0x7ff42c094a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c134220 0x7ff42c136610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c066870 con 0x7ff42c094f40 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c095ab0 con 0x7ff42c093de0 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff431394700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c06ab70 con 0x7ff42c134220 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff42affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c093de0 0x7ff42c094a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff42a7fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff42c094f40 0x7ff42c0953a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55626/0 (socket says 172.21.15.155:55626) 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff42a7fc700 1 -- 172.21.15.155:0/477798495 learned_addr learned my addr 172.21.15.155:0/477798495 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.343+0000 7ff42b7fe700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c134220 0x7ff42c136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff429ffb700 1 -- 172.21.15.155:0/477798495 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503200453 0 0) 0x7ff42c066870 con 0x7ff42c094f40 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff42b7fe700 1 -- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c093de0 msgr2=0x7ff42c094a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff42b7fe700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c093de0 0x7ff42c094a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff42b7fe700 1 -- 172.21.15.155:0/477798495 >> v1:172.21.15.155:6789/0 conn(0x7ff42c094f40 legacy=0x7ff42c0953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff42b7fe700 1 -- 172.21.15.155:0/477798495 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c096ba0 con 0x7ff42c134220 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff42affd700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c093de0 0x7ff42c094a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff42b7fe700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c134220 0x7ff42c136610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff424009dc0 tx=0x7ff42400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c88c274db37e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff429ffb700 1 -- 172.21.15.155:0/477798495 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400e070 con 0x7ff42c134220 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff429ffb700 1 -- 172.21.15.155:0/477798495 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42400bb20 con 0x7ff42c134220 2024-04-25T11:02:41.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.344+0000 7ff429ffb700 1 -- 172.21.15.155:0/477798495 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424012430 con 0x7ff42c134220 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 -- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c134220 msgr2=0x7ff42c136610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c134220 0x7ff42c136610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff424009dc0 tx=0x7ff42400b330 comp rx=0 tx=0).stop 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 -- 172.21.15.155:0/477798495 shutdown_connections 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c134220 0x7ff42c136610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 --2- 172.21.15.155:0/477798495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c093de0 0x7ff42c094a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 -- 172.21.15.155:0/477798495 >> 172.21.15.155:0/477798495 conn(0x7ff42c09d480 msgr2=0x7ff42c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 -- 172.21.15.155:0/477798495 shutdown_connections 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.345+0000 7ff431394700 1 -- 172.21.15.155:0/477798495 wait complete. 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 Processor -- start 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 -- start start 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c093de0 0x7ff42c1c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 0x7ff42c1c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff42c134220 0x7ff42c1c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c070000 con 0x7ff42c134220 2024-04-25T11:02:41.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0712d0 con 0x7ff42c094f40 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff431394700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c06f610 con 0x7ff42c093de0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff42a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 0x7ff42c1c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff42affd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c093de0 0x7ff42c1c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff42a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 0x7ff42c1c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41260/0 (socket says 172.21.15.155:41260) 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.346+0000 7ff42a7fc700 1 -- 172.21.15.155:0/2009846668 learned_addr learned my addr 172.21.15.155:0/2009846668 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42b7fe700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff42c134220 0x7ff42c1c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42affd700 1 -- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 msgr2=0x7ff42c1c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42affd700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 0x7ff42c1c3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42affd700 1 -- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff42c134220 msgr2=0x7ff42c1c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42affd700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff42c134220 0x7ff42c1c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42affd700 1 -- 172.21.15.155:0/2009846668 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c0094a0 con 0x7ff42c093de0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42a7fc700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 0x7ff42c1c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff42affd700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c093de0 0x7ff42c1c2f50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9b0 tx=0x7ff41c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00ee50 con 0x7ff42c093de0 2024-04-25T11:02:41.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c00c850 con 0x7ff42c093de0 2024-04-25T11:02:41.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff424009a70 con 0x7ff42c093de0 2024-04-25T11:02:41.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cc00 con 0x7ff42c093de0 2024-04-25T11:02:41.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.347+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c071590 con 0x7ff42c093de0 2024-04-25T11:02:41.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.348+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7ff41c013470 con 0x7ff42c093de0 2024-04-25T11:02:41.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.348+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c070680 con 0x7ff42c093de0 2024-04-25T11:02:41.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.349+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7ff41c05c580 con 0x7ff42c093de0 2024-04-25T11:02:41.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.356+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7ff41c03a410 con 0x7ff42c093de0 2024-04-25T11:02:41.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 Processor -- start 2024-04-25T11:02:41.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 -- start start 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd412ee40 0x7f6cd412f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd406f320 con 0x7f6cd406ebb0 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4072ad0 con 0x7f6cd412ee40 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.386+0000 7f6cd9b07700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd406ad30 con 0x7f6cd412f760 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd412ee40 0x7f6cd412f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd2ffd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f6cd406ebb0 0x7f6cd406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55636/0 (socket says 172.21.15.155:55636) 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd2ffd700 1 -- 172.21.15.155:0/2687255437 learned_addr learned my addr 172.21.15.155:0/2687255437 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd3fff700 1 --2- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd37fe700 1 -- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 msgr2=0x7f6cd406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd37fe700 1 --2- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd27fc700 1 -- 172.21.15.155:0/2687255437 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407886461 0 0) 0x7f6cd406f320 con 0x7f6cd406ebb0 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd37fe700 1 -- 172.21.15.155:0/2687255437 >> v1:172.21.15.155:6789/0 conn(0x7f6cd406ebb0 legacy=0x7f6cd406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd37fe700 1 -- 172.21.15.155:0/2687255437 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd406e1d0 con 0x7f6cd412ee40 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd37fe700 1 --2- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd412ee40 0x7f6cd412f220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009870 tx=0x7f6cc4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10a0955afb02597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd27fc700 1 -- 172.21.15.155:0/2687255437 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4011830 con 0x7f6cd412ee40 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd27fc700 1 -- 172.21.15.155:0/2687255437 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc4011990 con 0x7f6cd412ee40 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.387+0000 7f6cd27fc700 1 -- 172.21.15.155:0/2687255437 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4011d20 con 0x7f6cd412ee40 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 -- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd412ee40 msgr2=0x7f6cd412f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd412ee40 0x7f6cd412f220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009870 tx=0x7f6cc4009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 -- 172.21.15.155:0/2687255437 shutdown_connections 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/2687255437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd412ee40 0x7f6cd412f220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 -- 172.21.15.155:0/2687255437 >> 172.21.15.155:0/2687255437 conn(0x7f6cd409cd50 msgr2=0x7f6cd409d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 -- 172.21.15.155:0/2687255437 shutdown_connections 2024-04-25T11:02:41.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.388+0000 7f6cd9b07700 1 -- 172.21.15.155:0/2687255437 wait complete. 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.389+0000 7f6cd9b07700 1 Processor -- start 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.389+0000 7f6cd9b07700 1 -- start start 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 0x7f6cd4090420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd412ee40 0x7f6cd4090960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd4090ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40926e0 con 0x7f6cd412ee40 2024-04-25T11:02:41.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4092b40 con 0x7f6cd406ebb0 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4093ad0 con 0x7f6cd412f760 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 0x7f6cd4090420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 0x7f6cd4090420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41272/0 (socket says 172.21.15.155:41272) 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 -- 172.21.15.155:0/3733801035 learned_addr learned my addr 172.21.15.155:0/3733801035 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd2ffd700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd412ee40 0x7f6cd4090960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd3fff700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd4090ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 -- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 msgr2=0x7f6cd4090ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd4090ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 -- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd412ee40 msgr2=0x7f6cd4090960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd412ee40 0x7f6cd4090960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 -- 172.21.15.155:0/3733801035 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40094a0 con 0x7f6cd406ebb0 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd3fff700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd4090ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:41.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd37fe700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 0x7f6cd4090420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4000c00 tx=0x7f6cc4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4004590 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc401f070 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd4091dd0 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40038e0 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.390+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd41c3320 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.392+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd4070310 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.392+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f6cc4003a40 con 0x7f6cd406ebb0 2024-04-25T11:02:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.392+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f6cc4021070 con 0x7f6cd406ebb0 2024-04-25T11:02:41.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.396+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f6cc4027bb0 con 0x7f6cd406ebb0 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 Processor -- start 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 -- start start 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406d8b0 0x7f7ff406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 0x7f7ff412e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ff412ea60 0x7f7ff412ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff406ebc0 con 0x7f7ff412ea60 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff406e630 con 0x7f7ff406d8b0 2024-04-25T11:02:41.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff881e700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff412f450 con 0x7f7ff406e1d0 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.425+0000 7f7ff1d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 0x7f7ff412e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 0x7f7ff412e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:38974/0 (socket says 172.21.15.155:38974) 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 -- 172.21.15.155:0/3525379026 learned_addr learned my addr 172.21.15.155:0/3525379026 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff259c700 1 --2- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406d8b0 0x7f7ff406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff2d9d700 1 --2- 172.21.15.155:0/3525379026 >> v2:172.21.15.155:3300/0 conn(0x7f7ff412ea60 0x7f7ff412ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 -- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406d8b0 msgr2=0x7f7ff406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 --2- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406d8b0 0x7f7ff406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 -- 172.21.15.155:0/3525379026 >> v2:172.21.15.155:3300/0 conn(0x7f7ff412ea60 msgr2=0x7f7ff412ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 --2- 172.21.15.155:0/3525379026 >> v2:172.21.15.155:3300/0 conn(0x7f7ff412ea60 0x7f7ff412ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 -- 172.21.15.155:0/3525379026 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff406faa0 con 0x7f7ff406e1d0 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff2d9d700 1 --2- 172.21.15.155:0/3525379026 >> v2:172.21.15.155:3300/0 conn(0x7f7ff412ea60 0x7f7ff412ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:41.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff1d9b700 1 --2- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 0x7f7ff412e520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009920 tx=0x7f7fe8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b970e1da5fb124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff159a700 1 -- 172.21.15.155:0/3525379026 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8011830 con 0x7f7ff406e1d0 2024-04-25T11:02:41.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff159a700 1 -- 172.21.15.155:0/3525379026 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8011990 con 0x7f7ff406e1d0 2024-04-25T11:02:41.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.426+0000 7f7ff159a700 1 -- 172.21.15.155:0/3525379026 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8011d20 con 0x7f7ff406e1d0 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 -- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 msgr2=0x7f7ff412e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 0x7f7ff412e520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009920 tx=0x7f7fe8009c30 comp rx=0 tx=0).stop 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 -- 172.21.15.155:0/3525379026 shutdown_connections 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3525379026 >> v2:172.21.15.155:3300/0 conn(0x7f7ff412ea60 0x7f7ff412ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406d8b0 0x7f7ff406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3525379026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff406e1d0 0x7f7ff412e520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 -- 172.21.15.155:0/3525379026 >> 172.21.15.155:0/3525379026 conn(0x7f7ff409ecb0 msgr2=0x7f7ff409f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 -- 172.21.15.155:0/3525379026 shutdown_connections 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.427+0000 7f7ff881e700 1 -- 172.21.15.155:0/3525379026 wait complete. 2024-04-25T11:02:41.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 Processor -- start 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 -- start start 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff406d8b0 0x7f7ff41bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 0x7f7ff41bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff412ea60 0x7f7ff41c04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4135210 con 0x7f7ff406d8b0 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4134db0 con 0x7f7ff406e1d0 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff881e700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4135440 con 0x7f7ff412ea60 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff1d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 0x7f7ff41bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.428+0000 7f7ff1d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 0x7f7ff41bff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41288/0 (socket says 172.21.15.155:41288) 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff1d9b700 1 -- 172.21.15.155:0/3148774940 learned_addr learned my addr 172.21.15.155:0/3148774940 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff412ea60 0x7f7ff41c04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff259c700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff406d8b0 0x7f7ff41bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:41.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 -- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 msgr2=0x7f7ff41bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 0x7f7ff41bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 -- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff406d8b0 msgr2=0x7f7ff41bfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff406d8b0 0x7f7ff41bfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 -- 172.21.15.155:0/3148774940 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec009a70 con 0x7f7ff412ea60 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff259c700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff406d8b0 0x7f7ff41bfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff1d9b700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 0x7f7ff41bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff2d9d700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff412ea60 0x7f7ff41c04c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009d90 tx=0x7f7fec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00cb40 con 0x7f7ff412ea60 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec00f040 con 0x7f7ff412ea60 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec003a20 con 0x7f7ff412ea60 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8009550 con 0x7f7ff412ea60 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.429+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff41c2f40 con 0x7f7ff412ea60 2024-04-25T11:02:41.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.430+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4134830 con 0x7f7ff412ea60 2024-04-25T11:02:41.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.431+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f7fec014480 con 0x7f7ff412ea60 2024-04-25T11:02:41.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.431+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f7fec05cd50 con 0x7f7ff412ea60 2024-04-25T11:02:41.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.438+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f7fec03ab30 con 0x7f7ff412ea60 2024-04-25T11:02:41.534 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.533+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff42c062510 con 0x7ff42c093de0 2024-04-25T11:02:41.534 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.533+0000 7ff4137fe700 1 -- 172.21.15.155:0/2009846668 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff41c017030 con 0x7ff42c093de0 2024-04-25T11:02:41.534 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.535+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 >> v1:172.21.15.155:6816/49624 conn(0x7ff418047440 legacy=0x7ff418049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.535+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c093de0 msgr2=0x7ff42c1c2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.535+0000 7ff431394700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c093de0 0x7ff42c1c2f50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9b0 tx=0x7ff41c00ecc0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.535+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 shutdown_connections 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.535+0000 7ff431394700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff42c134220 0x7ff42c1c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.536+0000 7ff431394700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff42c093de0 0x7ff42c1c2f50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.536+0000 7ff431394700 1 --2- 172.21.15.155:0/2009846668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff42c094f40 0x7ff42c1c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.536+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 >> 172.21.15.155:0/2009846668 conn(0x7ff42c09d480 msgr2=0x7ff42c097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.536+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 shutdown_connections 2024-04-25T11:02:41.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.536+0000 7ff431394700 1 -- 172.21.15.155:0/2009846668 wait complete. 2024-04-25T11:02:41.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.3 2024-04-25T11:02:41.568 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.568+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6cd4062510 con 0x7f6cd406ebb0 2024-04-25T11:02:41.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.568+0000 7f6cbbfff700 1 -- 172.21.15.155:0/3733801035 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6cc4015020 con 0x7f6cd406ebb0 2024-04-25T11:02:41.569 INFO:teuthology.orchestra.run.smithi155.stdout:38654705685 2024-04-25T11:02:41.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 >> v1:172.21.15.155:6816/49624 conn(0x7f6cc0047440 legacy=0x7f6cc0049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 msgr2=0x7f6cd4090420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 0x7f6cd4090420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4000c00 tx=0x7f6cc4004750 comp rx=0 tx=0).stop 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 shutdown_connections 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cd412f760 0x7f6cd4090ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cd406ebb0 0x7f6cd4090420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 --2- 172.21.15.155:0/3733801035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd412ee40 0x7f6cd4090960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.570+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 >> 172.21.15.155:0/3733801035 conn(0x7f6cd409cd50 msgr2=0x7f6cd4099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.571+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 shutdown_connections 2024-04-25T11:02:41.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.571+0000 7f6cd9b07700 1 -- 172.21.15.155:0/3733801035 wait complete. 2024-04-25T11:02:41.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705685 for osd.0 2024-04-25T11:02:41.608 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.608+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ff4062510 con 0x7f7ff412ea60 2024-04-25T11:02:41.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.608+0000 7f7fe2ffd700 1 -- 172.21.15.155:0/3148774940 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7fec018030 con 0x7f7ff412ea60 2024-04-25T11:02:41.609 INFO:teuthology.orchestra.run.smithi155.stdout:42949672980 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 >> v1:172.21.15.155:6816/49624 conn(0x7f7fd4047440 legacy=0x7f7fd4049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff412ea60 msgr2=0x7f7ff41c04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff412ea60 0x7f7ff41c04c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009d90 tx=0x7f7fec00c3a0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 shutdown_connections 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7ff412ea60 0x7f7ff41c04c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff406d8b0 0x7f7ff41bfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 --2- 172.21.15.155:0/3148774940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ff406e1d0 0x7f7ff41bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 >> 172.21.15.155:0/3148774940 conn(0x7f7ff409ecb0 msgr2=0x7f7ff409b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 shutdown_connections 2024-04-25T11:02:41.611 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:41.610+0000 7f7ff881e700 1 -- 172.21.15.155:0/3148774940 wait complete. 2024-04-25T11:02:41.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672982 got 42949672980 for osd.1 2024-04-25T11:02:42.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:02:42.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.435+0000 7fc00e52f700 1 Processor -- start 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.435+0000 7fc00e52f700 1 -- start start 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00e52f700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00806eef0 0x7fc008133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00e52f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0081341e0 0x7fc0081365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00e52f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc008072780 con 0x7fc00806e400 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00e52f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00806ca20 con 0x7fc0081341e0 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00e52f700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc00806fb80 con 0x7fc00806eef0 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc0077fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00806eef0 0x7fc008133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00cacc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0081341e0 0x7fc0081365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc007fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc00806e400 0x7fc00806e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55652/0 (socket says 172.21.15.155:55652) 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc007fff700 1 -- 172.21.15.155:0/621741664 learned_addr learned my addr 172.21.15.155:0/621741664 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:42.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00cacc700 1 -- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00806eef0 msgr2=0x7fc008133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00cacc700 1 --2- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00806eef0 0x7fc008133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00cacc700 1 -- 172.21.15.155:0/621741664 >> v1:172.21.15.155:6789/0 conn(0x7fc00806e400 legacy=0x7fc00806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00cacc700 1 -- 172.21.15.155:0/621741664 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00806c6e0 con 0x7fc0081341e0 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc006ffd700 1 -- 172.21.15.155:0/621741664 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788112131 0 0) 0x7fc008072780 con 0x7fc00806e400 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc0077fe700 1 --2- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00806eef0 0x7fc008133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc00cacc700 1 --2- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0081341e0 0x7fc0081365d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc000009d90 tx=0x7fc00000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5415ee52874c7832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc006ffd700 1 -- 172.21.15.155:0/621741664 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000e070 con 0x7fc0081341e0 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc006ffd700 1 -- 172.21.15.155:0/621741664 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00000bb20 con 0x7fc0081341e0 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.436+0000 7fc006ffd700 1 -- 172.21.15.155:0/621741664 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000012430 con 0x7fc0081341e0 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 -- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0081341e0 msgr2=0x7fc0081365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 --2- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0081341e0 0x7fc0081365d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc000009d90 tx=0x7fc00000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 -- 172.21.15.155:0/621741664 shutdown_connections 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 --2- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0081341e0 0x7fc0081365d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 --2- 172.21.15.155:0/621741664 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc00806eef0 0x7fc008133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 -- 172.21.15.155:0/621741664 >> 172.21.15.155:0/621741664 conn(0x7fc00809dac0 msgr2=0x7fc00809ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 -- 172.21.15.155:0/621741664 shutdown_connections 2024-04-25T11:02:42.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.437+0000 7fc00e52f700 1 -- 172.21.15.155:0/621741664 wait complete. 2024-04-25T11:02:42.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 Processor -- start 2024-04-25T11:02:42.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 -- start start 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 0x7fc0081c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00806eef0 0x7fc0081c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc0081341e0 0x7fc0081c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080919e0 con 0x7fc00806e400 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc008133ca0 con 0x7fc00806eef0 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.438+0000 7fc00e52f700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc008133450 con 0x7fc0081341e0 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc007fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 0x7fc0081c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc0081341e0 0x7fc0081c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc007fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 0x7fc0081c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41654/0 (socket says 172.21.15.155:41654) 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc007fff700 1 -- 172.21.15.155:0/79752276 learned_addr learned my addr 172.21.15.155:0/79752276 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc0077fe700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00806eef0 0x7fc0081c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 -- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00806eef0 msgr2=0x7fc0081c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00806eef0 0x7fc0081c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 -- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 msgr2=0x7fc0081c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.444 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 0x7fc0081c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 -- 172.21.15.155:0/79752276 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00094a0 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc007fff700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 0x7fc0081c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00cacc700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc0081341e0 0x7fc0081c3910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc00806e020 tx=0x7fc000004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000012430 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000020070 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc000009a70 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00001bb80 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.439+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc008090c90 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.440+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0080908a0 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.440+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7fc00001bce0 con 0x7fc0081341e0 2024-04-25T11:02:42.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.441+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7fc00005d930 con 0x7fc0081341e0 2024-04-25T11:02:42.446 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.444+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fc000004370 con 0x7fc0081341e0 2024-04-25T11:02:42.549 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:02:42.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:02:42.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.608+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc008090c90 con 0x7fc0081341e0 2024-04-25T11:02:42.609 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.609+0000 7fc004ff9700 1 -- 172.21.15.155:0/79752276 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc008090c90 con 0x7fc0081341e0 2024-04-25T11:02:42.609 INFO:teuthology.orchestra.run.smithi155.stdout:42949672984 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 >> v1:172.21.15.155:6816/49624 conn(0x7fbfe8047440 legacy=0x7fbfe8049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc0081341e0 msgr2=0x7fc0081c3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc0081341e0 0x7fc0081c3910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc00806e020 tx=0x7fc000004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 shutdown_connections 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc0081341e0 0x7fc0081c3910 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806e400 0x7fc0081c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 --2- 172.21.15.155:0/79752276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc00806eef0 0x7fc0081c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 >> 172.21.15.155:0/79752276 conn(0x7fc00809dac0 msgr2=0x7fc0080710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.612 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 shutdown_connections 2024-04-25T11:02:42.613 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.611+0000 7fc00e52f700 1 -- 172.21.15.155:0/79752276 wait complete. 2024-04-25T11:02:42.623 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:02:42.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672983 got 42949672984 for osd.2 2024-04-25T11:02:42.626 DEBUG:teuthology.parallel:result is None 2024-04-25T11:02:42.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.677+0000 7f7a1f034700 1 Processor -- start 2024-04-25T11:02:42.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.677+0000 7f7a1f034700 1 -- start start 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1f034700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a1812cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1f034700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7a18070400 0x7f7a180708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1f034700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1812c5b0 con 0x7f7a1812c9e0 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1f034700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a18071e90 con 0x7f7a18070400 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1f034700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a181356b0 con 0x7f7a18071390 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a17fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a1812cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a17fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a1812cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39014/0 (socket says 172.21.15.155:39014) 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a17fff700 1 -- 172.21.15.155:0/3556839261 learned_addr learned my addr 172.21.15.155:0/3556839261 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:42.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1d5d1700 1 --2- 172.21.15.155:0/3556839261 >> v2:172.21.15.155:3300/0 conn(0x7f7a18070400 0x7f7a180708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1d5d1700 1 -- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 msgr2=0x7f7a1812cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:42.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1d5d1700 1 --2- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a1812cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1d5d1700 1 -- 172.21.15.155:0/3556839261 >> v1:172.21.15.155:6789/0 conn(0x7f7a1812c9e0 legacy=0x7f7a1806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a1d5d1700 1 -- 172.21.15.155:0/3556839261 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a181357f0 con 0x7f7a18070400 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a177fe700 1 -- 172.21.15.155:0/3556839261 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929507594 0 0) 0x7f7a1812c5b0 con 0x7f7a1812c9e0 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.678+0000 7f7a17fff700 1 --2- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a1812cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1d5d1700 1 --2- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18070400 0x7f7a180708b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009d90 tx=0x7f7a1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9943a5e26b7343ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a177fe700 1 -- 172.21.15.155:0/3556839261 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000e070 con 0x7f7a18070400 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a177fe700 1 -- 172.21.15.155:0/3556839261 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1000bbd0 con 0x7f7a18070400 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a177fe700 1 -- 172.21.15.155:0/3556839261 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10012430 con 0x7f7a18070400 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 -- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18070400 msgr2=0x7f7a180708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 --2- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18070400 0x7f7a180708b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009d90 tx=0x7f7a1000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 -- 172.21.15.155:0/3556839261 shutdown_connections 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 --2- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18070400 0x7f7a180708b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 --2- 172.21.15.155:0/3556839261 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a1812cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 -- 172.21.15.155:0/3556839261 >> 172.21.15.155:0/3556839261 conn(0x7f7a1809cd50 msgr2=0x7f7a1809d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.679+0000 7f7a1f034700 1 -- 172.21.15.155:0/3556839261 shutdown_connections 2024-04-25T11:02:42.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.680+0000 7f7a1f034700 1 -- 172.21.15.155:0/3556839261 wait complete. 2024-04-25T11:02:42.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.680+0000 7f7a1f034700 1 Processor -- start 2024-04-25T11:02:42.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.680+0000 7f7a1f034700 1 -- start start 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1f034700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 0x7f7a18132970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1f034700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a18132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1f034700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1812c9e0 0x7f7a181333f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1f034700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1806d810 con 0x7f7a1812c9e0 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1f034700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1806e960 con 0x7f7a18070400 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1f034700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a181342c0 con 0x7f7a18071390 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a17fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a18132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 0x7f7a18132970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1d5d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1812c9e0 0x7f7a181333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a17fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a18132eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39030/0 (socket says 172.21.15.155:39030) 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a17fff700 1 -- 172.21.15.155:0/1678102983 learned_addr learned my addr 172.21.15.155:0/1678102983 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 0x7f7a18132970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41302/0 (socket says 172.21.15.155:41302) 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 -- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 msgr2=0x7f7a18132eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a18132eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 -- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1812c9e0 msgr2=0x7f7a181333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1812c9e0 0x7f7a181333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1cdd0700 1 -- 172.21.15.155:0/1678102983 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a10009a70 con 0x7f7a18070400 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a1d5d1700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1812c9e0 0x7f7a181333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.681+0000 7f7a17fff700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a18132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.682+0000 7f7a1cdd0700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 0x7f7a18132970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c430 tx=0x7f7a0c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.682+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0054e0 con 0x7f7a18070400 2024-04-25T11:02:42.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.682+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00ce40 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.682+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a18133930 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.682+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0038c0 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.682+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1806e000 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.683+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f7a0c003a20 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.683+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a181274e0 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.683+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f7a0c05b910 con 0x7f7a18070400 2024-04-25T11:02:42.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.687+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f7a0c0397c0 con 0x7f7a18070400 2024-04-25T11:02:42.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 Processor -- start 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 -- start start 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb4f812c9e0 0x7fb4f812cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f812c5b0 con 0x7fb4f8070230 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f8071e90 con 0x7fb4f812c9e0 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.722+0000 7fb4fe652700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81356b0 con 0x7fb4f8071390 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f77fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f7fff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb4f812c9e0 0x7fb4f812cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4fcbef700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb4f8070230 0x7fb4f80706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55676/0 (socket says 172.21.15.155:55676) 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4fcbef700 1 -- 172.21.15.155:0/111296116 learned_addr learned my addr 172.21.15.155:0/111296116 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f7fff700 1 -- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 msgr2=0x7fb4f806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f7fff700 1 --2- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f7fff700 1 -- 172.21.15.155:0/111296116 >> v1:172.21.15.155:6789/0 conn(0x7fb4f8070230 legacy=0x7fb4f80706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f7fff700 1 -- 172.21.15.155:0/111296116 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f81357f0 con 0x7fb4f812c9e0 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f7fff700 1 --2- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f812c9e0 0x7fb4f812cdc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009840 tx=0x7fb4e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68534c11c802a4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f6ffd700 1 -- 172.21.15.155:0/111296116 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8011830 con 0x7fb4f812c9e0 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f6ffd700 1 -- 172.21.15.155:0/111296116 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8011990 con 0x7fb4f812c9e0 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.723+0000 7fb4f6ffd700 1 -- 172.21.15.155:0/111296116 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8011d20 con 0x7fb4f812c9e0 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 -- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f812c9e0 msgr2=0x7fb4f812cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 --2- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f812c9e0 0x7fb4f812cdc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009840 tx=0x7fb4e8009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 -- 172.21.15.155:0/111296116 shutdown_connections 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 --2- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f812c9e0 0x7fb4f812cdc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 --2- 172.21.15.155:0/111296116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 -- 172.21.15.155:0/111296116 >> 172.21.15.155:0/111296116 conn(0x7fb4f809cd50 msgr2=0x7fb4f809d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 -- 172.21.15.155:0/111296116 shutdown_connections 2024-04-25T11:02:42.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.724+0000 7fb4fe652700 1 -- 172.21.15.155:0/111296116 wait complete. 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.725+0000 7fb4fe652700 1 Processor -- start 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 -- start start 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8070230 0x7fb4f806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb4f812c9e0 0x7fb4f806e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f806f600 con 0x7fb4f8070230 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8132dd0 con 0x7fb4f8071390 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4fe652700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8134390 con 0x7fb4f812c9e0 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4f77fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4f7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8070230 0x7fb4f806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4f77fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41320/0 (socket says 172.21.15.155:41320) 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.726+0000 7fb4f77fe700 1 -- 172.21.15.155:0/4141104297 learned_addr learned my addr 172.21.15.155:0/4141104297 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4fcbef700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb4f812c9e0 0x7fb4f806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f77fe700 1 -- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb4f812c9e0 msgr2=0x7fb4f806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f77fe700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb4f812c9e0 0x7fb4f806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f77fe700 1 -- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8070230 msgr2=0x7fb4f806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f77fe700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8070230 0x7fb4f806d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f77fe700 1 -- 172.21.15.155:0/4141104297 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80094a0 con 0x7fb4f8071390 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f77fe700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806dac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00e980 tx=0x7fb4ec00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec009710 con 0x7fb4f8071390 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec00c690 con 0x7fb4f8071390 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f806e8a0 con 0x7fb4f8071390 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00ca20 con 0x7fb4f8071390 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.727+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f806f000 con 0x7fb4f8071390 2024-04-25T11:02:42.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.728+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f8062510 con 0x7fb4f8071390 2024-04-25T11:02:42.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.729+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7fb4ec00cb80 con 0x7fb4f8071390 2024-04-25T11:02:42.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.729+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7fb4ec05c590 con 0x7fb4f8071390 2024-04-25T11:02:42.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.733+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fb4ec03a480 con 0x7fb4f8071390 2024-04-25T11:02:42.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.751+0000 7f6cfb4c9700 1 Processor -- start 2024-04-25T11:02:42.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.751+0000 7f6cfb4c9700 1 -- start start 2024-04-25T11:02:42.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cfb4c9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4131b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cfb4c9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4134430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cfb4c9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf4071f10 con 0x7f6cf4097480 2024-04-25T11:02:42.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cfb4c9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4071d20 con 0x7f6cf412f710 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cfb4c9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4097f70 con 0x7f6cf4132040 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4131b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4131b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41322/0 (socket says 172.21.15.155:41322) 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 -- 172.21.15.155:0/3138502249 learned_addr learned my addr 172.21.15.155:0/3138502249 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf9a66700 1 --2- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 -- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 msgr2=0x7f6cf4134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 --2- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4134430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 -- 172.21.15.155:0/3138502249 >> v1:172.21.15.155:6789/0 conn(0x7f6cf4097480 legacy=0x7f6cf4097860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.752+0000 7f6cf8a64700 1 -- 172.21.15.155:0/3138502249 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40721d0 con 0x7f6cf412f710 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.753+0000 7f6cf8a64700 1 --2- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4131b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009920 tx=0x7f6ce4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db91eeef25771172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.753+0000 7f6cebfff700 1 -- 172.21.15.155:0/3138502249 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4011830 con 0x7f6cf412f710 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.753+0000 7f6cebfff700 1 -- 172.21.15.155:0/3138502249 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4011990 con 0x7f6cf412f710 2024-04-25T11:02:42.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.753+0000 7f6cebfff700 1 -- 172.21.15.155:0/3138502249 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4011d20 con 0x7f6cf412f710 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 msgr2=0x7f6cf4131b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4131b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009920 tx=0x7f6ce4009c30 comp rx=0 tx=0).stop 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/3138502249 shutdown_connections 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4134430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/3138502249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4131b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/3138502249 >> 172.21.15.155:0/3138502249 conn(0x7f6cf409dac0 msgr2=0x7f6cf409ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/3138502249 shutdown_connections 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.754+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/3138502249 wait complete. 2024-04-25T11:02:42.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 Processor -- start 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 -- start start 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cf4097480 0x7f6cf4126ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4127000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4127540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4127f50 con 0x7f6cf4097480 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4130440 con 0x7f6cf412f710 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cfb4c9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4130600 con 0x7f6cf4132040 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cf8a64700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4127000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cf8a64700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4127000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41328/0 (socket says 172.21.15.155:41328) 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cf8a64700 1 -- 172.21.15.155:0/1045839332 learned_addr learned my addr 172.21.15.155:0/1045839332 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cf9a66700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4127540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.755+0000 7f6cf9265700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cf4097480 0x7f6cf4126ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cf8a64700 1 -- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 msgr2=0x7f6cf4127540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cf8a64700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4127540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cf8a64700 1 -- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cf4097480 msgr2=0x7f6cf4126ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cf8a64700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cf4097480 0x7f6cf4126ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cf8a64700 1 -- 172.21.15.155:0/1045839332 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009550 con 0x7f6cf412f710 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cf8a64700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4127000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4004450 con 0x7f6cf412f710 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40045b0 con 0x7f6cf412f710 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf412bbc0 con 0x7f6cf412f710 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40038a0 con 0x7f6cf412f710 2024-04-25T11:02:42.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.756+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf412fe40 con 0x7f6cf412f710 2024-04-25T11:02:42.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.757+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 7) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f6ce4021070 con 0x7f6cf412f710 2024-04-25T11:02:42.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.757+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf4067020 con 0x7f6cf412f710 2024-04-25T11:02:42.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.758+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2967+0+0 (secure 0 0 0) 0x7f6ce405d790 con 0x7f6cf412f710 2024-04-25T11:02:42.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.765+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f6ce4026c00 con 0x7f6cf412f710 2024-04-25T11:02:42.867 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.867+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a18062510 con 0x7f7a18070400 2024-04-25T11:02:42.868 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.867+0000 7f7a157fa700 1 -- 172.21.15.155:0/1678102983 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a0c017030 con 0x7f7a18070400 2024-04-25T11:02:42.868 INFO:teuthology.orchestra.run.smithi155.stdout:38654705688 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 >> v1:172.21.15.155:6816/49624 conn(0x7f79fc047470 legacy=0x7f79fc049930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 msgr2=0x7f7a18132970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 0x7f7a18132970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c430 tx=0x7f7a0c00c740 comp rx=0 tx=0).stop 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 shutdown_connections 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1812c9e0 0x7f7a181333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a18070400 0x7f7a18132970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 --2- 172.21.15.155:0/1678102983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7a18071390 0x7f7a18132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.870 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 >> 172.21.15.155:0/1678102983 conn(0x7f7a1809cd50 msgr2=0x7f7a1812b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.871 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 shutdown_connections 2024-04-25T11:02:42.871 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.869+0000 7f7a1f034700 1 -- 172.21.15.155:0/1678102983 wait complete. 2024-04-25T11:02:42.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705688 for osd.3 2024-04-25T11:02:42.882 DEBUG:teuthology.parallel:result is None 2024-04-25T11:02:42.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.904+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb4f808b2a0 con 0x7fb4f8071390 2024-04-25T11:02:42.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.905+0000 7fb4f4ff9700 1 -- 172.21.15.155:0/4141104297 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb4f808b2a0 con 0x7fb4f8071390 2024-04-25T11:02:42.905 INFO:teuthology.orchestra.run.smithi155.stdout:38654705688 2024-04-25T11:02:42.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 >> v1:172.21.15.155:6816/49624 conn(0x7fb4e4047440 legacy=0x7fb4e4049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 msgr2=0x7fb4f806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806dac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00e980 tx=0x7fb4ec00ec90 comp rx=0 tx=0).stop 2024-04-25T11:02:42.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 shutdown_connections 2024-04-25T11:02:42.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb4f812c9e0 0x7fb4f806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8070230 0x7fb4f806d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 --2- 172.21.15.155:0/4141104297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4f8071390 0x7fb4f806dac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.906+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 >> 172.21.15.155:0/4141104297 conn(0x7fb4f809cd50 msgr2=0x7fb4f812ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.907+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 shutdown_connections 2024-04-25T11:02:42.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.907+0000 7fb4fe652700 1 -- 172.21.15.155:0/4141104297 wait complete. 2024-04-25T11:02:42.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705688 for osd.0 2024-04-25T11:02:42.919 DEBUG:teuthology.parallel:result is None 2024-04-25T11:02:42.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.935+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6cf412ec70 con 0x7f6cf412f710 2024-04-25T11:02:42.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.935+0000 7f6ce9ffb700 1 -- 172.21.15.155:0/1045839332 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ce4015030 con 0x7f6cf412f710 2024-04-25T11:02:42.935 INFO:teuthology.orchestra.run.smithi155.stdout:42949672983 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 >> v1:172.21.15.155:6816/49624 conn(0x7f6cd80474a0 legacy=0x7f6cd8049960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 msgr2=0x7f6cf4127000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4127000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce4004750 comp rx=0 tx=0).stop 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 shutdown_connections 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6cf4132040 0x7f6cf4127540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cf4097480 0x7f6cf4126ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 --2- 172.21.15.155:0/1045839332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cf412f710 0x7f6cf4127000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 >> 172.21.15.155:0/1045839332 conn(0x7f6cf409dac0 msgr2=0x7f6cf4098b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 shutdown_connections 2024-04-25T11:02:42.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:42.937+0000 7f6cfb4c9700 1 -- 172.21.15.155:0/1045839332 wait complete. 2024-04-25T11:02:42.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672982 got 42949672983 for osd.1 2024-04-25T11:02:42.949 DEBUG:teuthology.parallel:result is None 2024-04-25T11:02:42.949 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:02:42.949 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:02:43.108 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 Processor -- start 2024-04-25T11:02:43.109 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 -- start start 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 0x7ff8b0133c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b0136580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b006cfd0 con 0x7ff8b0072000 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b006d320 con 0x7ff8b0072af0 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.108+0000 7ff8b4d6d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b0093cb0 con 0x7ff8b0134190 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8add9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 0x7ff8b0133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b0136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8add9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 0x7ff8b0133c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41720/0 (socket says 172.21.15.155:41720) 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8ae59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff8b0072000 0x7ff8b00723e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55684/0 (socket says 172.21.15.155:55684) 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b0136580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41344/0 (socket says 172.21.15.155:41344) 2024-04-25T11:02:43.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8add9b700 1 -- 172.21.15.155:0/1770097051 learned_addr learned my addr 172.21.15.155:0/1770097051 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 -- 172.21.15.155:0/1770097051 >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 msgr2=0x7ff8b0133c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 --2- 172.21.15.155:0/1770097051 >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 0x7ff8b0133c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 -- 172.21.15.155:0/1770097051 >> v1:172.21.15.155:6789/0 conn(0x7ff8b0072000 legacy=0x7ff8b00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 -- 172.21.15.155:0/1770097051 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00921e0 con 0x7ff8b0134190 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8add9b700 1 --2- 172.21.15.155:0/1770097051 >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 0x7ff8b0133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8aed9d700 1 --2- 172.21.15.155:0/1770097051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b0136580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009d90 tx=0x7ff8a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0f95aea01b9a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8ad59a700 1 -- 172.21.15.155:0/1770097051 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800e070 con 0x7ff8b0134190 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8ad59a700 1 -- 172.21.15.155:0/1770097051 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a800bb20 con 0x7ff8b0134190 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.109+0000 7ff8ad59a700 1 -- 172.21.15.155:0/1770097051 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8012430 con 0x7ff8b0134190 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/1770097051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 msgr2=0x7ff8b0136580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/1770097051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b0136580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009d90 tx=0x7ff8a800b330 comp rx=0 tx=0).stop 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/1770097051 shutdown_connections 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/1770097051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b0136580 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/1770097051 >> v2:172.21.15.155:3300/0 conn(0x7ff8b0072af0 0x7ff8b0133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/1770097051 >> 172.21.15.155:0/1770097051 conn(0x7ff8b009a140 msgr2=0x7ff8b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:43.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/1770097051 shutdown_connections 2024-04-25T11:02:43.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.110+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/1770097051 wait complete. 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 Processor -- start 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 -- start start 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 0x7ff8b012c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 0x7ff8b012cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b012d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b012e790 con 0x7ff8b0072000 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b012ea30 con 0x7ff8b0134190 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8b4d6d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b008ff60 con 0x7ff8b0072af0 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8add9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 0x7ff8b012cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8ae59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 0x7ff8b012c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.115 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8aed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b012d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8add9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 0x7ff8b012cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39068/0 (socket says 172.21.15.155:39068) 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8ae59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 0x7ff8b012c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41732/0 (socket says 172.21.15.155:41732) 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.111+0000 7ff8add9b700 1 -- 172.21.15.155:0/2632116209 learned_addr learned my addr 172.21.15.155:0/2632116209 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8add9b700 1 -- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 msgr2=0x7ff8b012d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8add9b700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b012d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8add9b700 1 -- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 msgr2=0x7ff8b012c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8add9b700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 0x7ff8b012c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8add9b700 1 -- 172.21.15.155:0/2632116209 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a0009550 con 0x7ff8b0072af0 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8ae59c700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 0x7ff8b012c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8aed9d700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b012d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8add9b700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 0x7ff8b012cad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000ebe0 tx=0x7ff8a000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000c850 con 0x7ff8b0072af0 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a000c9b0 con 0x7ff8b0072af0 2024-04-25T11:02:43.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000cd40 con 0x7ff8b0072af0 2024-04-25T11:02:43.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a8009a70 con 0x7ff8b0072af0 2024-04-25T11:02:43.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.112+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b012dff0 con 0x7ff8b0072af0 2024-04-25T11:02:43.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.113+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b0062510 con 0x7ff8b0072af0 2024-04-25T11:02:43.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.113+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7ff8a0013470 con 0x7ff8b0072af0 2024-04-25T11:02:43.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.113+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2950+0+0 (secure 0 0 0) 0x7ff8a005c8f0 con 0x7ff8b0072af0 2024-04-25T11:02:43.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.117+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7ff8a003a5c0 con 0x7ff8b0072af0 2024-04-25T11:02:43.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.272+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8b012d780 con 0x7ff890047560 2024-04-25T11:02:43.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.274+0000 7ff89effd700 1 -- 172.21.15.155:0/2632116209 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+38871 (unknown 2965378022 0 3060704796) 0x7ff8b012d780 con 0x7ff890047560 2024-04-25T11:02:43.275 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:43.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 >> v1:172.21.15.155:6816/49624 conn(0x7ff890047560 legacy=0x7ff890049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 msgr2=0x7ff8b012cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 0x7ff8b012cad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000ebe0 tx=0x7ff8a000eef0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 shutdown_connections 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b0134190 0x7ff8b012d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0072000 0x7ff8b012c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 --2- 172.21.15.155:0/2632116209 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff8b0072af0 0x7ff8b012cad0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 >> 172.21.15.155:0/2632116209 conn(0x7ff8b009a140 msgr2=0x7ff8b0134ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 shutdown_connections 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.276+0000 7ff8b4d6d700 1 -- 172.21.15.155:0/2632116209 wait complete. 2024-04-25T11:02:43.277 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:02:43.288 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T11:02:42.988009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188884,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298476,"statfs":{"total":386547056640,"available":385329639424,"internally_reserved":0,"allocated":512000,"data_stored":142739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481151+0000","last_change":"2024-04-25T11:01:01.361046+0000","last_active":"2024-04-25T11:01:06.481151+0000","last_peered":"2024-04-25T11:01:06.481151+0000","last_clean":"2024-04-25T11:01:06.481151+0000","last_became_active":"2024-04-25T11:01:01.360746+0000","last_became_peered":"2024-04-25T11:01:01.360746+0000","last_unstale":"2024-04-25T11:01:06.481151+0000","last_undegraded":"2024-04-25T11:01:06.481151+0000","last_fullsized":"2024-04-25T11:01:06.481151+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481460+0000","last_change":"2024-04-25T11:01:01.361196+0000","last_active":"2024-04-25T11:01:06.481460+0000","last_peered":"2024-04-25T11:01:06.481460+0000","last_clean":"2024-04-25T11:01:06.481460+0000","last_became_active":"2024-04-25T11:01:01.360891+0000","last_became_peered":"2024-04-25T11:01:01.360891+0000","last_unstale":"2024-04-25T11:01:06.481460+0000","last_undegraded":"2024-04-25T11:01:06.481460+0000","last_fullsized":"2024-04-25T11:01:06.481460+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481049+0000","last_change":"2024-04-25T11:01:01.361055+0000","last_active":"2024-04-25T11:01:06.481049+0000","last_peered":"2024-04-25T11:01:06.481049+0000","last_clean":"2024-04-25T11:01:06.481049+0000","last_became_active":"2024-04-25T11:01:01.360774+0000","last_became_peered":"2024-04-25T11:01:01.360774+0000","last_unstale":"2024-04-25T11:01:06.481049+0000","last_undegraded":"2024-04-25T11:01:06.481049+0000","last_fullsized":"2024-04-25T11:01:06.481049+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.480932+0000","last_change":"2024-04-25T11:01:00.434886+0000","last_active":"2024-04-25T11:01:06.480932+0000","last_peered":"2024-04-25T11:01:06.480932+0000","last_clean":"2024-04-25T11:01:06.480932+0000","last_became_active":"2024-04-25T11:01:00.434536+0000","last_became_peered":"2024-04-25T11:01:00.434536+0000","last_unstale":"2024-04-25T11:01:06.480932+0000","last_undegraded":"2024-04-25T11:01:06.480932+0000","last_fullsized":"2024-04-25T11:01:06.480932+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481273+0000","last_change":"2024-04-25T11:00:59.424526+0000","last_active":"2024-04-25T11:01:06.481273+0000","last_peered":"2024-04-25T11:01:06.481273+0000","last_clean":"2024-04-25T11:01:06.481273+0000","last_became_active":"2024-04-25T11:00:59.424232+0000","last_became_peered":"2024-04-25T11:00:59.424232+0000","last_unstale":"2024-04-25T11:01:06.481273+0000","last_undegraded":"2024-04-25T11:01:06.481273+0000","last_fullsized":"2024-04-25T11:01:06.481273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.480933+0000","last_change":"2024-04-25T11:01:00.431767+0000","last_active":"2024-04-25T11:01:06.480933+0000","last_peered":"2024-04-25T11:01:06.480933+0000","last_clean":"2024-04-25T11:01:06.480933+0000","last_became_active":"2024-04-25T11:01:00.431464+0000","last_became_peered":"2024-04-25T11:01:00.431464+0000","last_unstale":"2024-04-25T11:01:06.480933+0000","last_undegraded":"2024-04-25T11:01:06.480933+0000","last_fullsized":"2024-04-25T11:01:06.480933+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:08.582180+0000","last_change":"2024-04-25T11:01:00.429186+0000","last_active":"2024-04-25T11:01:08.582180+0000","last_peered":"2024-04-25T11:01:08.582180+0000","last_clean":"2024-04-25T11:01:08.582180+0000","last_became_active":"2024-04-25T11:01:00.428901+0000","last_became_peered":"2024-04-25T11:01:00.428901+0000","last_unstale":"2024-04-25T11:01:08.582180+0000","last_undegraded":"2024-04-25T11:01:08.582180+0000","last_fullsized":"2024-04-25T11:01:08.582180+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.480934+0000","last_change":"2024-04-25T11:01:00.434904+0000","last_active":"2024-04-25T11:01:06.480934+0000","last_peered":"2024-04-25T11:01:06.480934+0000","last_clean":"2024-04-25T11:01:06.480934+0000","last_became_active":"2024-04-25T11:01:00.434505+0000","last_became_peered":"2024-04-25T11:01:00.434505+0000","last_unstale":"2024-04-25T11:01:06.480934+0000","last_undegraded":"2024-04-25T11:01:06.480934+0000","last_fullsized":"2024-04-25T11:01:06.480934+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481354+0000","last_change":"2024-04-25T11:01:01.363969+0000","last_active":"2024-04-25T11:01:06.481354+0000","last_peered":"2024-04-25T11:01:06.481354+0000","last_clean":"2024-04-25T11:01:06.481354+0000","last_became_active":"2024-04-25T11:01:01.363581+0000","last_became_peered":"2024-04-25T11:01:01.363581+0000","last_unstale":"2024-04-25T11:01:06.481354+0000","last_undegraded":"2024-04-25T11:01:06.481354+0000","last_fullsized":"2024-04-25T11:01:06.481354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":131072,"data_stored":36774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 11:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.254},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.379}]},{"osd":1,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.33600000000000002}]}]},{"osd":2,"up_from":10,"seq":42949672984,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":131072,"data_stored":36774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 11:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Apr 25 11:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.317}]},{"osd":3,"last update":"Thu Apr 25 11:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.249},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.30199999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705688,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297212,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074628,"statfs":{"total":96636764160,"available":96332419072,"internally_reserved":0,"allocated":118784,"data_stored":32417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"last":0.309},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.253}]}]},{"osd":1,"up_from":10,"seq":42949672983,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":131072,"data_stored":36774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 11:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Apr 25 11:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.308}]},{"osd":3,"last update":"Thu Apr 25 11:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.374}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:02:43.289 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:02:43.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 Processor -- start 2024-04-25T11:02:43.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 -- start start 2024-04-25T11:02:43.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd21006d0c0 0x7fd210131630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd210131b70 0x7fd210133f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21006c950 con 0x7fd2100974a0 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd210097f90 con 0x7fd21006d0c0 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd216cc4700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd21006c6e0 con 0x7fd210131b70 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd20ffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd21006d0c0 0x7fd210131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd215261700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd210131b70 0x7fd210133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd214a60700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd2100974a0 0x7fd210097880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55692/0 (socket says 172.21.15.155:55692) 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd214a60700 1 -- 172.21.15.155:0/1459848347 learned_addr learned my addr 172.21.15.155:0/1459848347 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd20ffff700 1 -- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd210131b70 msgr2=0x7fd210133f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd20ffff700 1 --2- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd210131b70 0x7fd210133f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd20ffff700 1 -- 172.21.15.155:0/1459848347 >> v1:172.21.15.155:6789/0 conn(0x7fd2100974a0 legacy=0x7fd210097880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd20ffff700 1 -- 172.21.15.155:0/1459848347 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210071d50 con 0x7fd21006d0c0 2024-04-25T11:02:43.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd20f7fe700 1 -- 172.21.15.155:0/1459848347 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200218251 0 0) 0x7fd204001110 con 0x7fd2100974a0 2024-04-25T11:02:43.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.408+0000 7fd215261700 1 --2- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd210131b70 0x7fd210133f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd20ffff700 1 --2- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd210131630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009840 tx=0x7fd1fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d55f1561563428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:43.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd20f7fe700 1 -- 172.21.15.155:0/1459848347 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc011830 con 0x7fd21006d0c0 2024-04-25T11:02:43.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd20f7fe700 1 -- 172.21.15.155:0/1459848347 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc011990 con 0x7fd21006d0c0 2024-04-25T11:02:43.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd20f7fe700 1 -- 172.21.15.155:0/1459848347 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc011d20 con 0x7fd21006d0c0 2024-04-25T11:02:43.413 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 -- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 msgr2=0x7fd210131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.413 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 --2- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd210131630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009840 tx=0x7fd1fc009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 -- 172.21.15.155:0/1459848347 shutdown_connections 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 --2- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd210131b70 0x7fd210133f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 --2- 172.21.15.155:0/1459848347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd210131630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 -- 172.21.15.155:0/1459848347 >> 172.21.15.155:0/1459848347 conn(0x7fd21009dc90 msgr2=0x7fd21009e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 -- 172.21.15.155:0/1459848347 shutdown_connections 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.409+0000 7fd216cc4700 1 -- 172.21.15.155:0/1459848347 wait complete. 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.410+0000 7fd216cc4700 1 Processor -- start 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.410+0000 7fd216cc4700 1 -- start start 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd21012a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 0x7fd21012aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd210131b70 0x7fd21012af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21012c670 con 0x7fd21006d0c0 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21012d950 con 0x7fd210131b70 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd21012c8f0 con 0x7fd2100974a0 2024-04-25T11:02:43.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd214a60700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd21012a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 0x7fd21012aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 0x7fd21012aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39082/0 (socket says 172.21.15.155:39082) 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 -- 172.21.15.155:0/3034760490 learned_addr learned my addr 172.21.15.155:0/3034760490 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd214a60700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd21012a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41748/0 (socket says 172.21.15.155:41748) 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd215261700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd210131b70 0x7fd21012af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 -- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd210131b70 msgr2=0x7fd21012af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd210131b70 0x7fd21012af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 -- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 msgr2=0x7fd21012a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd21012a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 -- 172.21.15.155:0/3034760490 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009a70 con 0x7fd2100974a0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd214a60700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd21012a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20ffff700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 0x7fd21012aa10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0045d0 con 0x7fd2100974a0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc01f070 con 0x7fd2100974a0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0038a0 con 0x7fd2100974a0 2024-04-25T11:02:43.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.411+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc0094a0 con 0x7fd2100974a0 2024-04-25T11:02:43.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.412+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2101c0e30 con 0x7fd2100974a0 2024-04-25T11:02:43.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.412+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fd1fc003a00 con 0x7fd2100974a0 2024-04-25T11:02:43.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.412+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210062510 con 0x7fd2100974a0 2024-04-25T11:02:43.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.413+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2950+0+0 (secure 0 0 0) 0x7fd1fc021070 con 0x7fd2100974a0 2024-04-25T11:02:43.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.416+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fd1fc03f060 con 0x7fd2100974a0 2024-04-25T11:02:43.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.575+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd21006b910 con 0x7fd1f4047590 2024-04-25T11:02:43.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.577+0000 7fd20d7fa700 1 -- 172.21.15.155:0/3034760490 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+38871 (unknown 2965378022 0 3060704796) 0x7fd21006b910 con 0x7fd1f4047590 2024-04-25T11:02:43.578 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:43.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 >> v1:172.21.15.155:6816/49624 conn(0x7fd1f4047590 legacy=0x7fd1f4049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 msgr2=0x7fd21012aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 0x7fd21012aa10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc004750 comp rx=0 tx=0).stop 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 shutdown_connections 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd210131b70 0x7fd21012af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d0c0 0x7fd21012a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 --2- 172.21.15.155:0/3034760490 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fd2100974a0 0x7fd21012aa10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 >> 172.21.15.155:0/3034760490 conn(0x7fd21009dc90 msgr2=0x7fd21009a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 shutdown_connections 2024-04-25T11:02:43.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.579+0000 7fd216cc4700 1 -- 172.21.15.155:0/3034760490 wait complete. 2024-04-25T11:02:43.581 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:02:43.591 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T11:02:42.988009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188884,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298476,"statfs":{"total":386547056640,"available":385329639424,"internally_reserved":0,"allocated":512000,"data_stored":142739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481151+0000","last_change":"2024-04-25T11:01:01.361046+0000","last_active":"2024-04-25T11:01:06.481151+0000","last_peered":"2024-04-25T11:01:06.481151+0000","last_clean":"2024-04-25T11:01:06.481151+0000","last_became_active":"2024-04-25T11:01:01.360746+0000","last_became_peered":"2024-04-25T11:01:01.360746+0000","last_unstale":"2024-04-25T11:01:06.481151+0000","last_undegraded":"2024-04-25T11:01:06.481151+0000","last_fullsized":"2024-04-25T11:01:06.481151+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481460+0000","last_change":"2024-04-25T11:01:01.361196+0000","last_active":"2024-04-25T11:01:06.481460+0000","last_peered":"2024-04-25T11:01:06.481460+0000","last_clean":"2024-04-25T11:01:06.481460+0000","last_became_active":"2024-04-25T11:01:01.360891+0000","last_became_peered":"2024-04-25T11:01:01.360891+0000","last_unstale":"2024-04-25T11:01:06.481460+0000","last_undegraded":"2024-04-25T11:01:06.481460+0000","last_fullsized":"2024-04-25T11:01:06.481460+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481049+0000","last_change":"2024-04-25T11:01:01.361055+0000","last_active":"2024-04-25T11:01:06.481049+0000","last_peered":"2024-04-25T11:01:06.481049+0000","last_clean":"2024-04-25T11:01:06.481049+0000","last_became_active":"2024-04-25T11:01:01.360774+0000","last_became_peered":"2024-04-25T11:01:01.360774+0000","last_unstale":"2024-04-25T11:01:06.481049+0000","last_undegraded":"2024-04-25T11:01:06.481049+0000","last_fullsized":"2024-04-25T11:01:06.481049+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.480932+0000","last_change":"2024-04-25T11:01:00.434886+0000","last_active":"2024-04-25T11:01:06.480932+0000","last_peered":"2024-04-25T11:01:06.480932+0000","last_clean":"2024-04-25T11:01:06.480932+0000","last_became_active":"2024-04-25T11:01:00.434536+0000","last_became_peered":"2024-04-25T11:01:00.434536+0000","last_unstale":"2024-04-25T11:01:06.480932+0000","last_undegraded":"2024-04-25T11:01:06.480932+0000","last_fullsized":"2024-04-25T11:01:06.480932+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481273+0000","last_change":"2024-04-25T11:00:59.424526+0000","last_active":"2024-04-25T11:01:06.481273+0000","last_peered":"2024-04-25T11:01:06.481273+0000","last_clean":"2024-04-25T11:01:06.481273+0000","last_became_active":"2024-04-25T11:00:59.424232+0000","last_became_peered":"2024-04-25T11:00:59.424232+0000","last_unstale":"2024-04-25T11:01:06.481273+0000","last_undegraded":"2024-04-25T11:01:06.481273+0000","last_fullsized":"2024-04-25T11:01:06.481273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.480933+0000","last_change":"2024-04-25T11:01:00.431767+0000","last_active":"2024-04-25T11:01:06.480933+0000","last_peered":"2024-04-25T11:01:06.480933+0000","last_clean":"2024-04-25T11:01:06.480933+0000","last_became_active":"2024-04-25T11:01:00.431464+0000","last_became_peered":"2024-04-25T11:01:00.431464+0000","last_unstale":"2024-04-25T11:01:06.480933+0000","last_undegraded":"2024-04-25T11:01:06.480933+0000","last_fullsized":"2024-04-25T11:01:06.480933+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:08.582180+0000","last_change":"2024-04-25T11:01:00.429186+0000","last_active":"2024-04-25T11:01:08.582180+0000","last_peered":"2024-04-25T11:01:08.582180+0000","last_clean":"2024-04-25T11:01:08.582180+0000","last_became_active":"2024-04-25T11:01:00.428901+0000","last_became_peered":"2024-04-25T11:01:00.428901+0000","last_unstale":"2024-04-25T11:01:08.582180+0000","last_undegraded":"2024-04-25T11:01:08.582180+0000","last_fullsized":"2024-04-25T11:01:08.582180+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.480934+0000","last_change":"2024-04-25T11:01:00.434904+0000","last_active":"2024-04-25T11:01:06.480934+0000","last_peered":"2024-04-25T11:01:06.480934+0000","last_clean":"2024-04-25T11:01:06.480934+0000","last_became_active":"2024-04-25T11:01:00.434505+0000","last_became_peered":"2024-04-25T11:01:00.434505+0000","last_unstale":"2024-04-25T11:01:06.480934+0000","last_undegraded":"2024-04-25T11:01:06.480934+0000","last_fullsized":"2024-04-25T11:01:06.480934+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-25T11:01:06.481354+0000","last_change":"2024-04-25T11:01:01.363969+0000","last_active":"2024-04-25T11:01:06.481354+0000","last_peered":"2024-04-25T11:01:06.481354+0000","last_clean":"2024-04-25T11:01:06.481354+0000","last_became_active":"2024-04-25T11:01:01.363581+0000","last_became_peered":"2024-04-25T11:01:01.363581+0000","last_unstale":"2024-04-25T11:01:06.481354+0000","last_undegraded":"2024-04-25T11:01:06.481354+0000","last_fullsized":"2024-04-25T11:01:06.481354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":131072,"data_stored":36774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 11:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.254},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.379}]},{"osd":1,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.33600000000000002}]}]},{"osd":2,"up_from":10,"seq":42949672984,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":131072,"data_stored":36774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 11:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Apr 25 11:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.317}]},{"osd":3,"last update":"Thu Apr 25 11:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.249},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.30199999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705688,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297212,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074628,"statfs":{"total":96636764160,"available":96332419072,"internally_reserved":0,"allocated":118784,"data_stored":32417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"last":0.309},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Apr 25 11:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.253}]}]},{"osd":1,"up_from":10,"seq":42949672983,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":131072,"data_stored":36774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 11:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Apr 25 11:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.308}]},{"osd":3,"last update":"Thu Apr 25 11:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.374}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:02:43.592 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:02:43.592 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:02:43.592 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:02:43.592 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.710+0000 7f48d9300700 1 Processor -- start 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.710+0000 7f48d9300700 1 -- start start 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d9300700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 0x7f48d412b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d9300700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f48d4128840 0x7f48d4133a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d9300700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d4093b40 con 0x7f48d412be00 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d9300700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d406d420 con 0x7f48d4128840 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d9300700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4072f70 con 0x7f48d412b4e0 2024-04-25T11:02:43.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d2ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 0x7f48d412b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f48d4128840 0x7f48d4133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d27fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f48d412be00 0x7f48d4128130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55702/0 (socket says 172.21.15.155:55702) 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f48d4128840 0x7f48d4133a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41752/0 (socket says 172.21.15.155:41752) 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d2ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 0x7f48d412b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39088/0 (socket says 172.21.15.155:39088) 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 -- 172.21.15.155:0/2777419038 learned_addr learned my addr 172.21.15.155:0/2777419038 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 -- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 msgr2=0x7f48d412b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 --2- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 0x7f48d412b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 -- 172.21.15.155:0/2777419038 >> v1:172.21.15.155:6789/0 conn(0x7f48d412be00 legacy=0x7f48d4128130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 -- 172.21.15.155:0/2777419038 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40932c0 con 0x7f48d4128840 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d1ffb700 1 -- 172.21.15.155:0/2777419038 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284503728 0 0) 0x7f48c4001110 con 0x7f48d412be00 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d2ffd700 1 --2- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 0x7f48d412b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d37fe700 1 --2- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4128840 0x7f48d4133a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009e50 tx=0x7f48cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8552c623f24db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:43.714 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d1ffb700 1 -- 172.21.15.155:0/2777419038 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00e070 con 0x7f48d4128840 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d1ffb700 1 -- 172.21.15.155:0/2777419038 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc00b910 con 0x7f48d4128840 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.711+0000 7f48d1ffb700 1 -- 172.21.15.155:0/2777419038 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00bcc0 con 0x7f48d4128840 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 -- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4128840 msgr2=0x7f48d4133a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 --2- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4128840 0x7f48d4133a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009e50 tx=0x7f48cc00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 -- 172.21.15.155:0/2777419038 shutdown_connections 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 --2- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4128840 0x7f48d4133a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 --2- 172.21.15.155:0/2777419038 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412b4e0 0x7f48d412b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 -- 172.21.15.155:0/2777419038 >> 172.21.15.155:0/2777419038 conn(0x7f48d409a140 msgr2=0x7f48d406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 -- 172.21.15.155:0/2777419038 shutdown_connections 2024-04-25T11:02:43.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.712+0000 7f48d9300700 1 -- 172.21.15.155:0/2777419038 wait complete. 2024-04-25T11:02:43.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 Processor -- start 2024-04-25T11:02:43.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 -- start start 2024-04-25T11:02:43.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48d4128840 0x7f48d41c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d412b4e0 0x7f48d41c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 0x7f48d4125bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4128200 con 0x7f48d412b4e0 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d408b9c0 con 0x7f48d4128840 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d9300700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48d408c270 con 0x7f48d412be00 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 0x7f48d4125bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.713+0000 7f48d2ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48d4128840 0x7f48d41c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 0x7f48d4125bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39102/0 (socket says 172.21.15.155:39102) 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d2ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48d4128840 0x7f48d41c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41370/0 (socket says 172.21.15.155:41370) 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 -- 172.21.15.155:0/1647014080 learned_addr learned my addr 172.21.15.155:0/1647014080 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d27fc700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d412b4e0 0x7f48d41c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 -- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48d4128840 msgr2=0x7f48d41c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48d4128840 0x7f48d41c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 -- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d412b4e0 msgr2=0x7f48d41c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d412b4e0 0x7f48d41c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 -- 172.21.15.155:0/1647014080 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80094a0 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d27fc700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d412b4e0 0x7f48d41c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d37fe700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 0x7f48d4125bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48cc004270 tx=0x7f48cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc0048a0 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc0092e0 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc009b30 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc01b650 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.714+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d41c40d0 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.715+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4062510 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.715+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f48cc01b7b0 con 0x7f48d412be00 2024-04-25T11:02:43.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.715+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2950+0+0 (secure 0 0 0) 0x7f48cc05dab0 con 0x7f48d412be00 2024-04-25T11:02:43.720 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.719+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f48cc03f210 con 0x7f48d412be00 2024-04-25T11:02:43.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.907+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f48d40928a0 con 0x7f48d412be00 2024-04-25T11:02:43.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.907+0000 7f48bb7fe700 1 -- 172.21.15.155:0/1647014080 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f48cc01f020 con 0x7f48d412be00 2024-04-25T11:02:43.908 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:43.908 INFO:teuthology.orchestra.run.smithi155.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 >> v1:172.21.15.155:6816/49624 conn(0x7f48c0047590 legacy=0x7f48c0049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 msgr2=0x7f48d4125bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 0x7f48d4125bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48cc004270 tx=0x7f48cc004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 shutdown_connections 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f48d412be00 0x7f48d4125bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48d4128840 0x7f48d41c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 --2- 172.21.15.155:0/1647014080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d412b4e0 0x7f48d41c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:43.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 >> 172.21.15.155:0/1647014080 conn(0x7f48d409a140 msgr2=0x7f48d4094510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:43.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 shutdown_connections 2024-04-25T11:02:43.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:43.909+0000 7f48d9300700 1 -- 172.21.15.155:0/1647014080 wait complete. 2024-04-25T11:02:43.922 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:02:43.922 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-04-25T11:02:43.931 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:02:43.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:43.963 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:02:43.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-04-25T11:02:44.127 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.124+0000 7ff355f09700 1 Processor -- start 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.124+0000 7ff355f09700 1 -- start start 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff355f09700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff3501286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff355f09700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 0x7ff35012b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff355f09700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff35012bb80 0x7ff3501325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff355f09700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3500728f0 con 0x7ff35012bb80 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff355f09700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35006f960 con 0x7ff3501282e0 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff355f09700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff350128c00 con 0x7ff35012b220 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34effd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 0x7ff35012b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff35012bb80 0x7ff3501325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff3501286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34effd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 0x7ff35012b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39112/0 (socket says 172.21.15.155:39112) 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34effd700 1 -- 172.21.15.155:0/2700642053 learned_addr learned my addr 172.21.15.155:0/2700642053 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:44.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff35012bb80 0x7ff3501325b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41774/0 (socket says 172.21.15.155:41774) 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 -- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 msgr2=0x7ff35012b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 0x7ff35012b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 -- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 msgr2=0x7ff3501286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff3501286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 -- 172.21.15.155:0/2700642053 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35006fe90 con 0x7ff35012bb80 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34f7fe700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff3501286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34effd700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 0x7ff35012b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34ffff700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012bb80 0x7ff3501325b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff348009dc0 tx=0x7ff34800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36f7ad36a17877a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34e7fc700 1 -- 172.21.15.155:0/2700642053 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800e070 con 0x7ff35012bb80 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34e7fc700 1 -- 172.21.15.155:0/2700642053 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34800bae0 con 0x7ff35012bb80 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.125+0000 7ff34e7fc700 1 -- 172.21.15.155:0/2700642053 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800be90 con 0x7ff35012bb80 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 -- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012bb80 msgr2=0x7ff3501325b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012bb80 0x7ff3501325b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff348009dc0 tx=0x7ff34800b330 comp rx=0 tx=0).stop 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 -- 172.21.15.155:0/2700642053 shutdown_connections 2024-04-25T11:02:44.129 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012bb80 0x7ff3501325b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff3501286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 --2- 172.21.15.155:0/2700642053 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012b220 0x7ff35012b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 -- 172.21.15.155:0/2700642053 >> 172.21.15.155:0/2700642053 conn(0x7ff35009dac0 msgr2=0x7ff35009ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 -- 172.21.15.155:0/2700642053 shutdown_connections 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.126+0000 7ff355f09700 1 -- 172.21.15.155:0/2700642053 wait complete. 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 Processor -- start 2024-04-25T11:02:44.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 -- start start 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff35008b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012b220 0x7ff35008b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 0x7ff35008ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35008d220 con 0x7ff35012b220 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35008c900 con 0x7ff3501282e0 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff355f09700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff350127020 con 0x7ff35012bb80 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 0x7ff35008ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff34f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff35008b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff34effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012b220 0x7ff35008b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.127+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 0x7ff35008ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39116/0 (socket says 172.21.15.155:39116) 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff35008b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41380/0 (socket says 172.21.15.155:41380) 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 -- 172.21.15.155:0/647001931 learned_addr learned my addr 172.21.15.155:0/647001931 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 -- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 msgr2=0x7ff35008b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff35008b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 -- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012b220 msgr2=0x7ff35008b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012b220 0x7ff35008b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 -- 172.21.15.155:0/647001931 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3440094a0 con 0x7ff35012bb80 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34effd700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012b220 0x7ff35008b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34f7fe700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff35008b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff34ffff700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 0x7ff35008ba80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff348014040 tx=0x7ff348004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:44.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348004550 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff348020070 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34000b7f0 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348004910 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.128+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff348009a70 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.129+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7ff348027480 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.129+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff350099a80 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.129+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2961+0+0 (secure 0 0 0) 0x7ff348022070 con 0x7ff35012bb80 2024-04-25T11:02:44.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.133+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7ff348009840 con 0x7ff35012bb80 2024-04-25T11:02:44.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.304+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7ff35008d080 con 0x7ff35012bb80 2024-04-25T11:02:44.305 INFO:tasks.ceph.mon.a.smithi155.stderr:2024-04-25T11:02:44.305+0000 7f1d61995700 -1 mon.a@0(leader).osd e17 definitely_dead 0 2024-04-25T11:02:44.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.305+0000 7ff337fff700 1 -- 172.21.15.155:0/647001931 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 osd.0 is already down. v17) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff35008d080 con 0x7ff35012bb80 2024-04-25T11:02:44.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 >> v1:172.21.15.155:6816/49624 conn(0x7ff33c047560 legacy=0x7ff33c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 msgr2=0x7ff35008ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 0x7ff35008ba80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff348014040 tx=0x7ff348004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 shutdown_connections 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff35012bb80 0x7ff35008ba80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3501282e0 0x7ff35008b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 --2- 172.21.15.155:0/647001931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35012b220 0x7ff35008b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 >> 172.21.15.155:0/647001931 conn(0x7ff35009dac0 msgr2=0x7ff350097700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 shutdown_connections 2024-04-25T11:02:44.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.307+0000 7ff355f09700 1 -- 172.21.15.155:0/647001931 wait complete. 2024-04-25T11:02:44.309 INFO:teuthology.orchestra.run.smithi155.stderr:osd.0 is already down. 2024-04-25T11:02:44.319 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T11:02:44.319 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T11:02:44.322 INFO:tasks.ceph.osd.0:Started 2024-04-25T11:02:44.322 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:02:44.322 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:44.353 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:02:44.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-04-25T11:02:44.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.513+0000 7f6958966700 1 Processor -- start 2024-04-25T11:02:44.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.513+0000 7f6958966700 1 -- start start 2024-04-25T11:02:44.516 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6958966700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406e400 0x7f695406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6958966700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 0x7f695412ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6958966700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 0x7f695412efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6958966700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695406ca20 con 0x7f695406ed20 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6958966700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6954072780 con 0x7f695406e400 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6958966700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f695406fb80 con 0x7f695406f140 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6951d9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 0x7f695412ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 0x7f695412efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f695259c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406e400 0x7f695406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 0x7f695412efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39124/0 (socket says 172.21.15.155:39124) 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6951d9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 0x7f695412ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41798/0 (socket says 172.21.15.155:41798) 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 -- 172.21.15.155:0/2958464588 learned_addr learned my addr 172.21.15.155:0/2958464588 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 -- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406e400 msgr2=0x7f695406e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:44.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 --2- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406e400 0x7f695406e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 -- 172.21.15.155:0/2958464588 >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 msgr2=0x7f695412ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 --2- 172.21.15.155:0/2958464588 >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 0x7f695412ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6952d9d700 1 -- 172.21.15.155:0/2958464588 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695406c6e0 con 0x7f695406f140 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f6951d9b700 1 --2- 172.21.15.155:0/2958464588 >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 0x7f695412ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.514+0000 7f695259c700 1 --2- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406e400 0x7f695406e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6952d9d700 1 --2- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 0x7f695412efa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f694c009e80 tx=0x7f694c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122c52416ee79fba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f695159a700 1 -- 172.21.15.155:0/2958464588 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00e070 con 0x7f695406f140 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f695159a700 1 -- 172.21.15.155:0/2958464588 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c00b8d0 con 0x7f695406f140 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f695159a700 1 -- 172.21.15.155:0/2958464588 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00bc80 con 0x7f695406f140 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 -- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 msgr2=0x7f695412efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 --2- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 0x7f695412efa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f694c009e80 tx=0x7f694c00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 -- 172.21.15.155:0/2958464588 shutdown_connections 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 --2- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406f140 0x7f695412efa0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 --2- 172.21.15.155:0/2958464588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406e400 0x7f695406e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 --2- 172.21.15.155:0/2958464588 >> v2:172.21.15.155:3300/0 conn(0x7f695406ed20 0x7f695412ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 -- 172.21.15.155:0/2958464588 >> 172.21.15.155:0/2958464588 conn(0x7f695409dac0 msgr2=0x7f695409ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:44.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 -- 172.21.15.155:0/2958464588 shutdown_connections 2024-04-25T11:02:44.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.515+0000 7f6958966700 1 -- 172.21.15.155:0/2958464588 wait complete. 2024-04-25T11:02:44.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 Processor -- start 2024-04-25T11:02:44.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 -- start start 2024-04-25T11:02:44.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 0x7f6954095b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406ed20 0x7f69540960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406f140 0x7f6954096600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695412f3a0 con 0x7f695406e400 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6954093bc0 con 0x7f695406f140 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6958966700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f695412efa0 con 0x7f695406ed20 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f695259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 0x7f6954095b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6951d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406ed20 0x7f69540960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f695259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 0x7f6954095b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41808/0 (socket says 172.21.15.155:41808) 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f695259c700 1 -- 172.21.15.155:0/3556580080 learned_addr learned my addr 172.21.15.155:0/3556580080 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.517+0000 7f6952d9d700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406f140 0x7f6954096600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f695259c700 1 -- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406ed20 msgr2=0x7f69540960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f695259c700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406ed20 0x7f69540960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f695259c700 1 -- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406f140 msgr2=0x7f6954096600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f695259c700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406f140 0x7f6954096600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f695259c700 1 -- 172.21.15.155:0/3556580080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440094a0 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f6951d9b700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406ed20 0x7f69540960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f695259c700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 0x7f6954095b80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f694400ec10 tx=0x7f694400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694400c850 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694400c9b0 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480094a0 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694400cd20 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.518+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c009b30 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.519+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f6944013470 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.519+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69541348f0 con 0x7f695406e400 2024-04-25T11:02:44.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.519+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f694405c860 con 0x7f695406e400 2024-04-25T11:02:44.524 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.523+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f694403a520 con 0x7f695406e400 2024-04-25T11:02:44.696 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.695+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7f6954062510 con 0x7f695406e400 2024-04-25T11:02:44.696 INFO:tasks.ceph.mon.a.smithi155.stderr:2024-04-25T11:02:44.695+0000 7f1d61995700 -1 mon.a@0(leader).osd e17 definitely_dead 0 2024-04-25T11:02:44.910 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:02:44.909+0000 7ff0d7a867c0 -1 Falling back to public interface 2024-04-25T11:02:44.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.990+0000 7f6942ffd700 1 -- 172.21.15.155:0/3556580080 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v18) v1 ==== 89+0+0 (secure 0 0 0) 0x7f6944017030 con 0x7f695406e400 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.992+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 >> v1:172.21.15.155:6816/49624 conn(0x7f6934047560 legacy=0x7f6934049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.992+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 msgr2=0x7f6954095b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.992+0000 7f6958966700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 0x7f6954095b80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f694400ec10 tx=0x7f694400ef20 comp rx=0 tx=0).stop 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.992+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 shutdown_connections 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.993+0000 7f6958966700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695406f140 0x7f6954096600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.993+0000 7f6958966700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695406e400 0x7f6954095b80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.993+0000 7f6958966700 1 --2- 172.21.15.155:0/3556580080 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f695406ed20 0x7f69540960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:44.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.993+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 >> 172.21.15.155:0/3556580080 conn(0x7f695409dac0 msgr2=0x7f6954131a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:44.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.993+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 shutdown_connections 2024-04-25T11:02:44.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:44.993+0000 7f6958966700 1 -- 172.21.15.155:0/3556580080 wait complete. 2024-04-25T11:02:44.995 INFO:teuthology.orchestra.run.smithi155.stderr:marked down osd.1. 2024-04-25T11:02:45.008 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T11:02:45.008 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T11:02:45.010 INFO:tasks.ceph.osd.1:Started 2024-04-25T11:02:45.010 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:02:45.010 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:45.047 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:02:45.047 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 Processor -- start 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 -- start start 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4001282e0 0x7fe4001286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 0x7fe4001335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000728f0 con 0x7fe40012b220 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40006f960 con 0x7fe4001282e0 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe4072af700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe400128c00 con 0x7fe40012bd80 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe40504b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4001282e0 0x7fe4001286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe40584c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 0x7fe4001335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:45.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe40504b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4001282e0 0x7fe4001286c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41832/0 (socket says 172.21.15.155:41832) 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe40584c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 0x7fe4001335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41402/0 (socket says 172.21.15.155:41402) 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.209+0000 7fe40504b700 1 -- 172.21.15.155:0/3824753050 learned_addr learned my addr 172.21.15.155:0/3824753050 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe40504b700 1 -- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 msgr2=0x7fe4001335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe40504b700 1 --2- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 0x7fe4001335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe40504b700 1 -- 172.21.15.155:0/3824753050 >> v1:172.21.15.155:6789/0 conn(0x7fe40012b220 legacy=0x7fe40012b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe40504b700 1 -- 172.21.15.155:0/3824753050 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40006fe90 con 0x7fe4001282e0 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe40584c700 1 --2- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 0x7fe4001335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe40504b700 1 --2- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4001282e0 0x7fe4001286c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0098f0 tx=0x7fe3fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368941e88067c1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe3f7fff700 1 -- 172.21.15.155:0/3824753050 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00f8a0 con 0x7fe4001282e0 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe3f7fff700 1 -- 172.21.15.155:0/3824753050 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc00fa00 con 0x7fe4001282e0 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.210+0000 7fe3f7fff700 1 -- 172.21.15.155:0/3824753050 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00fd90 con 0x7fe4001282e0 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 -- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4001282e0 msgr2=0x7fe4001286c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 --2- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4001282e0 0x7fe4001286c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0098f0 tx=0x7fe3fc009c00 comp rx=0 tx=0).stop 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 -- 172.21.15.155:0/3824753050 shutdown_connections 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 --2- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4001282e0 0x7fe4001286c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:45.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 --2- 172.21.15.155:0/3824753050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012bd80 0x7fe4001335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:45.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 -- 172.21.15.155:0/3824753050 >> 172.21.15.155:0/3824753050 conn(0x7fe40009dac0 msgr2=0x7fe40009ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:45.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 -- 172.21.15.155:0/3824753050 shutdown_connections 2024-04-25T11:02:45.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 -- 172.21.15.155:0/3824753050 wait complete. 2024-04-25T11:02:45.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.211+0000 7fe4072af700 1 Processor -- start 2024-04-25T11:02:45.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 -- start start 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 0x7fe40008bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 0x7fe40008c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 0x7fe40008c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40008ecf0 con 0x7fe40012bd80 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40008e730 con 0x7fe40012b220 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe4072af700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe40008ef90 con 0x7fe4001282e0 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40484a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 0x7fe40008c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40504b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 0x7fe40008bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:45.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 0x7fe40008c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40504b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 0x7fe40008bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39164/0 (socket says 172.21.15.155:39164) 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 0x7fe40008c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41834/0 (socket says 172.21.15.155:41834) 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40504b700 1 -- 172.21.15.155:0/799827659 learned_addr learned my addr 172.21.15.155:0/799827659 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40484a700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 0x7fe40008c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41404/0 (socket says 172.21.15.155:41404) 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 -- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 msgr2=0x7fe40008bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 0x7fe40008bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 -- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 msgr2=0x7fe40008c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 0x7fe40008c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 -- 172.21.15.155:0/799827659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009a70 con 0x7fe40012bd80 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40504b700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 0x7fe40008bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40484a700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 0x7fe40008c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.212+0000 7fe40584c700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 0x7fe40008c730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800efc0 tx=0x7fe3f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.213+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800ca90 con 0x7fe40012bd80 2024-04-25T11:02:45.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.213+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f800f040 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.213+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc009550 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.213+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800ce30 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.213+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40008dba0 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.214+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fe3f8014480 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.214+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe400062510 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.214+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fe3f805ccd0 con 0x7fe40012bd80 2024-04-25T11:02:45.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.218+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fe3f803a990 con 0x7fe40012bd80 2024-04-25T11:02:45.391 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:45.391+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7fe40006b910 con 0x7fe40012bd80 2024-04-25T11:02:45.392 INFO:tasks.ceph.mon.a.smithi155.stderr:2024-04-25T11:02:45.391+0000 7f1d61995700 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-04-25T11:02:45.599 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:02:45.598+0000 7f07e738d7c0 -1 Falling back to public interface 2024-04-25T11:02:46.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.008+0000 7fe3f5ffb700 1 -- 172.21.15.155:0/799827659 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v19) v1 ==== 89+0+0 (secure 0 0 0) 0x7fe3f8018030 con 0x7fe40012bd80 2024-04-25T11:02:46.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 >> v1:172.21.15.155:6816/49624 conn(0x7fe3e4047590 legacy=0x7fe3e4049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:46.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 msgr2=0x7fe40008c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:46.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 0x7fe40008c730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800efc0 tx=0x7fe3f800c3a0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 shutdown_connections 2024-04-25T11:02:46.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe40012bd80 0x7fe40008c730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe4001282e0 0x7fe40008bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 --2- 172.21.15.155:0/799827659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe40012b220 0x7fe40008c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 >> 172.21.15.155:0/799827659 conn(0x7fe40009dac0 msgr2=0x7fe4000984a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:46.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 shutdown_connections 2024-04-25T11:02:46.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.011+0000 7fe4072af700 1 -- 172.21.15.155:0/799827659 wait complete. 2024-04-25T11:02:46.013 INFO:teuthology.orchestra.run.smithi155.stderr:marked down osd.2. 2024-04-25T11:02:46.026 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T11:02:46.027 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T11:02:46.029 INFO:tasks.ceph.osd.2:Started 2024-04-25T11:02:46.029 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:02:46.029 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:02:46.058 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:02:46.058 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 Processor -- start 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 -- start start 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06e400 0x7fb56c06e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 0x7fb56c12ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 0x7fb56c12efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c06ca20 con 0x7fb56c06ed20 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c072780 con 0x7fb56c06e400 2024-04-25T11:02:46.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb570d0c700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c06fb80 con 0x7fb56c06f140 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb56ad9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 0x7fb56c12efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb569d9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 0x7fb56c12ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.219+0000 7fb56a59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06e400 0x7fb56c06e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56ad9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 0x7fb56c12efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39184/0 (socket says 172.21.15.155:39184) 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56ad9d700 1 -- 172.21.15.155:0/2278187596 learned_addr learned my addr 172.21.15.155:0/2278187596 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb569d9b700 1 --2- 172.21.15.155:0/2278187596 >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 0x7fb56c12ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41838/0 (socket says 172.21.15.155:41838) 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56a59c700 1 -- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 msgr2=0x7fb56c12efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56a59c700 1 --2- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 0x7fb56c12efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56a59c700 1 -- 172.21.15.155:0/2278187596 >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 msgr2=0x7fb56c12ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:46.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56a59c700 1 --2- 172.21.15.155:0/2278187596 >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 0x7fb56c12ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56a59c700 1 -- 172.21.15.155:0/2278187596 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c06c6e0 con 0x7fb56c06e400 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56ad9d700 1 --2- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 0x7fb56c12efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb569d9b700 1 --2- 172.21.15.155:0/2278187596 >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 0x7fb56c12ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56a59c700 1 --2- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06e400 0x7fb56c06e7e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009870 tx=0x7fb55c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4994c224648c1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56959a700 1 -- 172.21.15.155:0/2278187596 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011830 con 0x7fb56c06e400 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56959a700 1 -- 172.21.15.155:0/2278187596 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c011990 con 0x7fb56c06e400 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.220+0000 7fb56959a700 1 -- 172.21.15.155:0/2278187596 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011d20 con 0x7fb56c06e400 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 -- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06e400 msgr2=0x7fb56c06e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 --2- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06e400 0x7fb56c06e7e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009870 tx=0x7fb55c009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 -- 172.21.15.155:0/2278187596 shutdown_connections 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 --2- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06f140 0x7fb56c12efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 --2- 172.21.15.155:0/2278187596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06e400 0x7fb56c06e7e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.225 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 --2- 172.21.15.155:0/2278187596 >> v2:172.21.15.155:3300/0 conn(0x7fb56c06ed20 0x7fb56c12ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.225 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 -- 172.21.15.155:0/2278187596 >> 172.21.15.155:0/2278187596 conn(0x7fb56c09dac0 msgr2=0x7fb56c09ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:46.225 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 -- 172.21.15.155:0/2278187596 shutdown_connections 2024-04-25T11:02:46.225 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.221+0000 7fb570d0c700 1 -- 172.21.15.155:0/2278187596 wait complete. 2024-04-25T11:02:46.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 Processor -- start 2024-04-25T11:02:46.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 -- start start 2024-04-25T11:02:46.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 0x7fb56c095c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:46.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06ed20 0x7fb56c096150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:46.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06f140 0x7fb56c096690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:46.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c12f3a0 con 0x7fb56c06e400 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c094e30 con 0x7fb56c06f140 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb570d0c700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c094530 con 0x7fb56c06ed20 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb56a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 0x7fb56c095c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb56ad9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06f140 0x7fb56c096690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb569d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06ed20 0x7fb56c096150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb56a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 0x7fb56c095c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41850/0 (socket says 172.21.15.155:41850) 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.222+0000 7fb56a59c700 1 -- 172.21.15.155:0/1971032826 learned_addr learned my addr 172.21.15.155:0/1971032826 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56a59c700 1 -- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06ed20 msgr2=0x7fb56c096150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56a59c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06ed20 0x7fb56c096150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56a59c700 1 -- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06f140 msgr2=0x7fb56c096690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56a59c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06f140 0x7fb56c096690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56a59c700 1 -- 172.21.15.155:0/1971032826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5600094a0 con 0x7fb56c06e400 2024-04-25T11:02:46.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56ad9d700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06f140 0x7fb56c096690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb569d9b700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06ed20 0x7fb56c096150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb56a59c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 0x7fb56c095c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb55c000c00 tx=0x7fb55c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c004240 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c01f070 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb564009b30 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0045c0 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.223+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c0094a0 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.224+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fb55c021030 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.224+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c067020 con 0x7fb56c06e400 2024-04-25T11:02:46.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.224+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fb55c05d6d0 con 0x7fb56c06e400 2024-04-25T11:02:46.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.228+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fb55c03ed80 con 0x7fb56c06e400 2024-04-25T11:02:46.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:46.401+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7fb56c062510 con 0x7fb56c06e400 2024-04-25T11:02:46.403 INFO:tasks.ceph.mon.a.smithi155.stderr:2024-04-25T11:02:46.402+0000 7f1d61995700 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-04-25T11:02:46.627 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:02:46.627+0000 7f2eee7357c0 -1 Falling back to public interface 2024-04-25T11:02:47.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.011+0000 7fb55affd700 1 -- 172.21.15.155:0/1971032826 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v20) v1 ==== 89+0+0 (secure 0 0 0) 0x7fb55c03e990 con 0x7fb56c06e400 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 >> v1:172.21.15.155:6816/49624 conn(0x7fb54c047560 legacy=0x7fb54c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 msgr2=0x7fb56c095c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 0x7fb56c095c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb55c000c00 tx=0x7fb55c004750 comp rx=0 tx=0).stop 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 shutdown_connections 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c06f140 0x7fb56c096690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c06e400 0x7fb56c095c10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 --2- 172.21.15.155:0/1971032826 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb56c06ed20 0x7fb56c096150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 >> 172.21.15.155:0/1971032826 conn(0x7fb56c09dac0 msgr2=0x7fb56c131a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 shutdown_connections 2024-04-25T11:02:47.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.013+0000 7fb570d0c700 1 -- 172.21.15.155:0/1971032826 wait complete. 2024-04-25T11:02:47.015 INFO:teuthology.orchestra.run.smithi155.stderr:marked down osd.3. 2024-04-25T11:02:47.026 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T11:02:47.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T11:02:47.028 INFO:tasks.ceph.osd.3:Started 2024-04-25T11:02:47.029 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:02:47.029 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:47.151 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.150+0000 7f9563905700 1 Processor -- start 2024-04-25T11:02:47.151 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.150+0000 7f9563905700 1 -- start start 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9563905700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 0x7f955c133e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9563905700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 0x7f955c1367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9563905700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0726b0 con 0x7f955c12ebd0 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9563905700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06f5b0 con 0x7f955c1343c0 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9563905700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06c6e0 con 0x7f955c12f6c0 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9561ea2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 0x7f955c1367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 0x7f955c133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:47.154 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 0x7f955c133e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39204/0 (socket says 172.21.15.155:39204) 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9561ea2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 0x7f955c1367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41454/0 (socket says 172.21.15.155:41454) 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 -- 172.21.15.155:0/2214547686 learned_addr learned my addr 172.21.15.155:0/2214547686 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 -- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 msgr2=0x7f955c1367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 --2- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 0x7f955c1367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f955bfff700 1 -- 172.21.15.155:0/2214547686 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522448985 0 0) 0x7f955c0726b0 con 0x7f955c12ebd0 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 -- 172.21.15.155:0/2214547686 >> v1:172.21.15.155:6789/0 conn(0x7f955c12ebd0 legacy=0x7f955c12efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 -- 172.21.15.155:0/2214547686 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c06fb70 con 0x7f955c12f6c0 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9561ea2700 1 --2- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 0x7f955c1367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f9560ea0700 1 --2- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 0x7f955c133e80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9550009950 tx=0x7f9550009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76e9cb7a02f037b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f955bfff700 1 -- 172.21.15.155:0/2214547686 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550011830 con 0x7f955c12f6c0 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f955bfff700 1 -- 172.21.15.155:0/2214547686 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550011990 con 0x7f955c12f6c0 2024-04-25T11:02:47.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.151+0000 7f955bfff700 1 -- 172.21.15.155:0/2214547686 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550011d40 con 0x7f955c12f6c0 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 -- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 msgr2=0x7f955c133e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 --2- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 0x7f955c133e80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9550009950 tx=0x7f9550009c60 comp rx=0 tx=0).stop 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 -- 172.21.15.155:0/2214547686 shutdown_connections 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 --2- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c1343c0 0x7f955c1367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 --2- 172.21.15.155:0/2214547686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c12f6c0 0x7f955c133e80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 -- 172.21.15.155:0/2214547686 >> 172.21.15.155:0/2214547686 conn(0x7f955c09dbc0 msgr2=0x7f955c09dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 -- 172.21.15.155:0/2214547686 shutdown_connections 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.152+0000 7f9563905700 1 -- 172.21.15.155:0/2214547686 wait complete. 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.153+0000 7f9563905700 1 Processor -- start 2024-04-25T11:02:47.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.153+0000 7f9563905700 1 -- start start 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9563905700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c12ebd0 0x7f955c090860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9563905700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 0x7f955c090da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9563905700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c1343c0 0x7f955c1c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9563905700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c093240 con 0x7f955c12f6c0 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9563905700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c092900 con 0x7f955c12ebd0 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9563905700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f955c091780 con 0x7f955c1343c0 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 0x7f955c090da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9561ea2700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c1343c0 0x7f955c1c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f95616a1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c12ebd0 0x7f955c090860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 0x7f955c090da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41862/0 (socket says 172.21.15.155:41862) 2024-04-25T11:02:47.159 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 -- 172.21.15.155:0/2759698239 learned_addr learned my addr 172.21.15.155:0/2759698239 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9561ea2700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c1343c0 0x7f955c1c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:39210/0 (socket says 172.21.15.155:39210) 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 -- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c1343c0 msgr2=0x7f955c1c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c1343c0 0x7f955c1c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 -- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c12ebd0 msgr2=0x7f955c090860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c12ebd0 0x7f955c090860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 -- 172.21.15.155:0/2759698239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c009a70 con 0x7f955c12f6c0 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9560ea0700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 0x7f955c090da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9550009130 tx=0x7f9550004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.154+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955001a840 con 0x7f955c12f6c0 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.155+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955001f070 con 0x7f955c12f6c0 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.155+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550009550 con 0x7f955c12f6c0 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.155+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955001ac00 con 0x7f955c12f6c0 2024-04-25T11:02:47.160 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.155+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c0912e0 con 0x7f955c12f6c0 2024-04-25T11:02:47.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.156+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c092a70 con 0x7f955c12f6c0 2024-04-25T11:02:47.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.156+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f955001b430 con 0x7f955c12f6c0 2024-04-25T11:02:47.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.156+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f9550021070 con 0x7f955c12f6c0 2024-04-25T11:02:47.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.160+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f9550027eb0 con 0x7f955c12f6c0 2024-04-25T11:02:47.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.326+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f955c067080 con 0x7f955c12f6c0 2024-04-25T11:02:47.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.327+0000 7f9559ffb700 1 -- 172.21.15.155:0/2759698239 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+8358 (secure 0 0 0) 0x7f955002b070 con 0x7f955c12f6c0 2024-04-25T11:02:47.328 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:47.328 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":20,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:02:47.002020+0000","last_up_change":"2024-04-25T11:02:47.002020+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":43900}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":43900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":43900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":43900}]},"public_addr":"172.21.15.155:6800/43900","cluster_addr":"172.21.15.155:6801/43900","heartbeat_back_addr":"172.21.15.155:6803/43900","heartbeat_front_addr":"172.21.15.155:6802/43900","state":["exists"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":17},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":18},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":19},{"osd":3,"down_stamp":"2024-04-25T11:02:47.002020+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:0/3077341019":"2024-04-26T11:02:39.971273+0000","172.21.15.155:0/966131479":"2024-04-26T11:02:39.971273+0000","172.21.15.155:6816/43298":"2024-04-26T11:02:39.971273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 >> v1:172.21.15.155:6816/49624 conn(0x7f9548047560 legacy=0x7f9548049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 msgr2=0x7f955c090da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 0x7f955c090da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9550009130 tx=0x7f9550004750 comp rx=0 tx=0).stop 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 shutdown_connections 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f955c1343c0 0x7f955c1c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f955c12ebd0 0x7f955c090860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 --2- 172.21.15.155:0/2759698239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f955c12f6c0 0x7f955c090da0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 >> 172.21.15.155:0/2759698239 conn(0x7f955c09dbc0 msgr2=0x7f955c0704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 shutdown_connections 2024-04-25T11:02:47.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:47.329+0000 7f9563905700 1 -- 172.21.15.155:0/2759698239 wait complete. 2024-04-25T11:02:47.627 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:02:47.625+0000 7fb7e36b07c0 -1 Falling back to public interface 2024-04-25T11:02:48.362 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:02:48.362+0000 7ff0d7a867c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-25T11:02:48.378 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:02:48.374+0000 7ff0d7a867c0 -1 osd.0 16 log_to_monitors true 2024-04-25T11:02:49.192 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:02:49.191+0000 7f07e738d7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-25T11:02:49.206 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:02:49.206+0000 7f07e738d7c0 -1 osd.1 17 log_to_monitors true 2024-04-25T11:02:49.882 INFO:tasks.ceph.osd.0.smithi155.stderr:2024-04-25T11:02:49.881+0000 7ff0c3a02700 -1 osd.0 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:02:50.209 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:02:50.209+0000 7f2eee7357c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-25T11:02:50.223 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:02:50.222+0000 7f2eee7357c0 -1 osd.2 17 log_to_monitors true 2024-04-25T11:02:50.342 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:50.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 Processor -- start 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 -- start start 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406e8e0 0x7ff10406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 0x7ff10412e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff10412ef30 0x7ff10412f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff104097f90 con 0x7ff10412ef30 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040700e0 con 0x7ff10406f200 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff10a9f8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff10406fe70 con 0x7ff10406e8e0 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff1037fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 0x7ff10412e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff1037fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 0x7ff10412e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41476/0 (socket says 172.21.15.155:41476) 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff1037fe700 1 -- 172.21.15.155:0/2894993503 learned_addr learned my addr 172.21.15.155:0/2894993503 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff103fff700 1 --2- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406e8e0 0x7ff10406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff108f95700 1 --2- 172.21.15.155:0/2894993503 >> v2:172.21.15.155:3300/0 conn(0x7ff10412ef30 0x7ff10412f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:50.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.486+0000 7ff1037fe700 1 -- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406e8e0 msgr2=0x7ff10406ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff1037fe700 1 --2- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406e8e0 0x7ff10406ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff1037fe700 1 -- 172.21.15.155:0/2894993503 >> v2:172.21.15.155:3300/0 conn(0x7ff10412ef30 msgr2=0x7ff10412f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff1037fe700 1 --2- 172.21.15.155:0/2894993503 >> v2:172.21.15.155:3300/0 conn(0x7ff10412ef30 0x7ff10412f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff1037fe700 1 -- 172.21.15.155:0/2894993503 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10406c740 con 0x7ff10406f200 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff1037fe700 1 --2- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 0x7ff10412e9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009920 tx=0x7ff0f8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1697e64642bf801 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff102ffd700 1 -- 172.21.15.155:0/2894993503 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8011830 con 0x7ff10406f200 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff102ffd700 1 -- 172.21.15.155:0/2894993503 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8011990 con 0x7ff10406f200 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff102ffd700 1 -- 172.21.15.155:0/2894993503 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8011d20 con 0x7ff10406f200 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 msgr2=0x7ff10412e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 0x7ff10412e9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009920 tx=0x7ff0f8009c30 comp rx=0 tx=0).stop 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.487+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2894993503 shutdown_connections 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2894993503 >> v2:172.21.15.155:3300/0 conn(0x7ff10412ef30 0x7ff10412f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406e8e0 0x7ff10406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2894993503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406f200 0x7ff10412e9f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2894993503 >> 172.21.15.155:0/2894993503 conn(0x7ff10409dc90 msgr2=0x7ff10409e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:50.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2894993503 shutdown_connections 2024-04-25T11:02:50.491 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2894993503 wait complete. 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.488+0000 7ff10a9f8700 1 Processor -- start 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 -- start start 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 0x7ff1041c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406f200 0x7ff1041c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff10412ef30 0x7ff1041c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104123ab0 con 0x7ff10412ef30 2024-04-25T11:02:50.493 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff104123860 con 0x7ff10406e8e0 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff10a9f8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1041351d0 con 0x7ff10406f200 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff103fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 0x7ff1041c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff1037fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406f200 0x7ff1041c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff103fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 0x7ff1041c2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41482/0 (socket says 172.21.15.155:41482) 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff103fff700 1 -- 172.21.15.155:0/2165454618 learned_addr learned my addr 172.21.15.155:0/2165454618 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.489+0000 7ff108f95700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff10412ef30 0x7ff1041c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff108f95700 1 -- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406f200 msgr2=0x7ff1041c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff108f95700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406f200 0x7ff1041c3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff108f95700 1 -- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 msgr2=0x7ff1041c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff108f95700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 0x7ff1041c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff108f95700 1 -- 172.21.15.155:0/2165454618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8009550 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff103fff700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 0x7ff1041c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff108f95700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff10412ef30 0x7ff1041c39d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00efc0 tx=0x7ff0fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00cc90 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00f040 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff104123fb0 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0146c0 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.490+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff104134170 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.491+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff104062510 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.492+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7ff0fc014820 con 0x7ff10412ef30 2024-04-25T11:02:50.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.492+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 2961+0+0 (secure 0 0 0) 0x7ff0fc05d690 con 0x7ff10412ef30 2024-04-25T11:02:50.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.496+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7ff0fc03b3e0 con 0x7ff10412ef30 2024-04-25T11:02:50.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.730+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff10406b910 con 0x7ff10412ef30 2024-04-25T11:02:50.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.731+0000 7ff100ff9700 1 -- 172.21.15.155:0/2165454618 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+8766 (secure 0 0 0) 0x7ff0fc02a090 con 0x7ff10412ef30 2024-04-25T11:02:50.732 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:50.732 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":22,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:02:50.018075+0000","last_up_change":"2024-04-25T11:02:50.018075+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.0), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":16,"up_from":22,"up_thru":0,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":51324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":51324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":51324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":51324}]},"public_addr":"172.21.15.155:6800/51324","cluster_addr":"172.21.15.155:6801/51324","heartbeat_back_addr":"172.21.15.155:6803/51324","heartbeat_front_addr":"172.21.15.155:6802/51324","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":43899}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":43899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":43899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":43899}]},"public_addr":"172.21.15.155:6808/43899","cluster_addr":"172.21.15.155:6809/43899","heartbeat_back_addr":"172.21.15.155:6811/43899","heartbeat_front_addr":"172.21.15.155:6810/43899","state":["exists"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":43902}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":43902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":43902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":43902}]},"public_addr":"172.21.15.155:6812/43902","cluster_addr":"172.21.15.155:6813/43902","heartbeat_back_addr":"172.21.15.155:6815/43902","heartbeat_front_addr":"172.21.15.155:6814/43902","state":["exists"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":43901}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":43901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":43901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":43901}]},"public_addr":"172.21.15.155:6804/43901","cluster_addr":"172.21.15.155:6805/43901","heartbeat_back_addr":"172.21.15.155:6807/43901","heartbeat_front_addr":"172.21.15.155:6806/43901","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":17},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":18},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":19},{"osd":3,"down_stamp":"2024-04-25T11:02:47.002020+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:0/3077341019":"2024-04-26T11:02:39.971273+0000","172.21.15.155:0/966131479":"2024-04-26T11:02:39.971273+0000","172.21.15.155:6816/43298":"2024-04-26T11:02:39.971273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:50.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 >> v1:172.21.15.155:6816/49624 conn(0x7ff0f0047560 legacy=0x7ff0f0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:50.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff10412ef30 msgr2=0x7ff1041c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:50.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff10412ef30 0x7ff1041c39d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00efc0 tx=0x7ff0fc00c3a0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 shutdown_connections 2024-04-25T11:02:50.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff10412ef30 0x7ff1041c39d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff10406e8e0 0x7ff1041c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 --2- 172.21.15.155:0/2165454618 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7ff10406f200 0x7ff1041c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:50.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 >> 172.21.15.155:0/2165454618 conn(0x7ff10409dc90 msgr2=0x7ff104099de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:50.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 shutdown_connections 2024-04-25T11:02:50.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:50.733+0000 7ff10a9f8700 1 -- 172.21.15.155:0/2165454618 wait complete. 2024-04-25T11:02:50.923 INFO:tasks.ceph.osd.1.smithi155.stderr:2024-04-25T11:02:50.922+0000 7f07d3309700 -1 osd.1 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:02:51.435 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:02:51.434+0000 7fb7e36b07c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-25T11:02:51.448 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:02:51.447+0000 7fb7e36b07c0 -1 osd.3 19 log_to_monitors true 2024-04-25T11:02:51.934 INFO:tasks.ceph.osd.2.smithi155.stderr:2024-04-25T11:02:51.933+0000 7f2eda6b1700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:02:52.893 INFO:tasks.ceph.osd.3.smithi155.stderr:2024-04-25T11:02:52.893+0000 7fb7cf62c700 -1 osd.3 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:02:53.750 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.877+0000 7fed6e982700 1 Processor -- start 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.877+0000 7fed6e982700 1 -- start start 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6e982700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed6812ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6e982700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fed6812b860 0x7fed68136990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6e982700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6812f5a0 con 0x7fed6812f180 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6e982700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6806fc80 con 0x7fed6812b860 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6e982700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed6806f8d0 con 0x7fed6806e650 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed67fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed6812ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6cf1f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fed6812b860 0x7fed68136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:53.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed677fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fed6812f180 0x7fed6812b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35284/0 (socket says 172.21.15.155:35284) 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed677fe700 1 -- 172.21.15.155:0/1861208962 learned_addr learned my addr 172.21.15.155:0/1861208962 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6cf1f700 1 -- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 msgr2=0x7fed6812ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6cf1f700 1 --2- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed6812ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6cf1f700 1 -- 172.21.15.155:0/1861208962 >> v1:172.21.15.155:6789/0 conn(0x7fed6812f180 legacy=0x7fed6812b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6cf1f700 1 -- 172.21.15.155:0/1861208962 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6806f0d0 con 0x7fed6812b860 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed67fff700 1 --2- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed6812ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed6cf1f700 1 --2- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed68136990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed60009d90 tx=0x7fed6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c160137b8fc12b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed66ffd700 1 -- 172.21.15.155:0/1861208962 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6000e070 con 0x7fed6812b860 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed66ffd700 1 -- 172.21.15.155:0/1861208962 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6000bb20 con 0x7fed6812b860 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.878+0000 7fed66ffd700 1 -- 172.21.15.155:0/1861208962 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60012430 con 0x7fed6812b860 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 -- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 msgr2=0x7fed68136990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 --2- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed68136990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed60009d90 tx=0x7fed6000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 -- 172.21.15.155:0/1861208962 shutdown_connections 2024-04-25T11:02:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 --2- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed68136990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:53.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 --2- 172.21.15.155:0/1861208962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed6812ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:53.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 -- 172.21.15.155:0/1861208962 >> 172.21.15.155:0/1861208962 conn(0x7fed6809b8c0 msgr2=0x7fed6809bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:53.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 -- 172.21.15.155:0/1861208962 shutdown_connections 2024-04-25T11:02:53.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.879+0000 7fed6e982700 1 -- 172.21.15.155:0/1861208962 wait complete. 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 Processor -- start 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 -- start start 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed68090460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed680909a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed6812f180 0x7fed68090ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68091f30 con 0x7fed6812b860 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed68093320 con 0x7fed6812f180 2024-04-25T11:02:53.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed6e982700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed681288a0 con 0x7fed6806e650 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed677fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed680909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.880+0000 7fed67fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed68090460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed680909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40414/0 (socket says 172.21.15.155:40414) 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed6cf1f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed6812f180 0x7fed68090ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 -- 172.21.15.155:0/2346415393 learned_addr learned my addr 172.21.15.155:0/2346415393 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed67fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed68090460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44240/0 (socket says 172.21.15.155:44240) 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 -- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 msgr2=0x7fed68090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed68090460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 -- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed6812f180 msgr2=0x7fed68090ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed6812f180 0x7fed68090ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 -- 172.21.15.155:0/2346415393 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed580094a0 con 0x7fed6812b860 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed677fe700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed680909a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00c500 tx=0x7fed5c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c005d70 con 0x7fed6812b860 2024-04-25T11:02:53.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c0070d0 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c007440 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed60009a70 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.881+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed680929c0 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.882+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed68062510 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.882+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fed5c0075a0 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.882+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fed5c05b990 con 0x7fed6812b860 2024-04-25T11:02:53.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:53.886+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fed5c0396b0 con 0x7fed6812b860 2024-04-25T11:02:54.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.055+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed6812f5a0 con 0x7fed6812b860 2024-04-25T11:02:54.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.056+0000 7fed64ff9700 1 -- 172.21.15.155:0/2346415393 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+8813 (secure 0 0 0) 0x7fed6812f5a0 con 0x7fed6812b860 2024-04-25T11:02:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:54.057 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":26,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:02:54.040550+0000","last_up_change":"2024-04-25T11:02:53.034844+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":16,"up_from":22,"up_thru":22,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":51324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":51324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":51324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":51324}]},"public_addr":"172.21.15.155:6800/51324","cluster_addr":"172.21.15.155:6801/51324","heartbeat_back_addr":"172.21.15.155:6803/51324","heartbeat_front_addr":"172.21.15.155:6802/51324","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":23,"up_thru":23,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":51418}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":51418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":51418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":51418}]},"public_addr":"172.21.15.155:6808/51418","cluster_addr":"172.21.15.155:6809/51418","heartbeat_back_addr":"172.21.15.155:6811/51418","heartbeat_front_addr":"172.21.15.155:6810/51418","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":24,"up_thru":25,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":51529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":51529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":51529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":51529}]},"public_addr":"172.21.15.155:6804/51529","cluster_addr":"172.21.15.155:6805/51529","heartbeat_back_addr":"172.21.15.155:6807/51529","heartbeat_front_addr":"172.21.15.155:6806/51529","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":51677}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":51677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":51677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":51677}]},"public_addr":"172.21.15.155:6812/51677","cluster_addr":"172.21.15.155:6813/51677","heartbeat_back_addr":"172.21.15.155:6815/51677","heartbeat_front_addr":"172.21.15.155:6814/51677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":17},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":18},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":19},{"osd":3,"down_stamp":"2024-04-25T11:02:47.002020+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:0/3077341019":"2024-04-26T11:02:39.971273+0000","172.21.15.155:0/966131479":"2024-04-26T11:02:39.971273+0000","172.21.15.155:6816/43298":"2024-04-26T11:02:39.971273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:54.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 >> v1:172.21.15.155:6816/49624 conn(0x7fed54047560 legacy=0x7fed54049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 msgr2=0x7fed680909a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed680909a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00c500 tx=0x7fed5c00c810 comp rx=0 tx=0).stop 2024-04-25T11:02:54.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 shutdown_connections 2024-04-25T11:02:54.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed6812f180 0x7fed68090ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fed6806e650 0x7fed68090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 --2- 172.21.15.155:0/2346415393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6812b860 0x7fed680909a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 >> 172.21.15.155:0/2346415393 conn(0x7fed6809b8c0 msgr2=0x7fed6809f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:54.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 shutdown_connections 2024-04-25T11:02:54.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.058+0000 7fed6e982700 1 -- 172.21.15.155:0/2346415393 wait complete. 2024-04-25T11:02:54.071 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:02:54.071 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T11:02:54.081 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:02:54.081 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:02:54.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2024-04-25T11:02:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.196+0000 7f6e2f928700 1 Processor -- start 2024-04-25T11:02:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.196+0000 7f6e2f928700 1 -- start start 2024-04-25T11:02:54.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2f928700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 0x7f6e2812ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2f928700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 0x7f6e2812b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2f928700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2f928700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e28098620 con 0x7f6e2812e6a0 2024-04-25T11:02:54.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2f928700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2812c0b0 con 0x7f6e2812b6e0 2024-04-25T11:02:54.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2f928700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2806de70 con 0x7f6e2812efc0 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2d6c4700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 0x7f6e2812ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2cec3700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 0x7f6e2812b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2d6c4700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 0x7f6e2812ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40420/0 (socket says 172.21.15.155:40420) 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40484/0 (socket says 172.21.15.155:40484) 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2cec3700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 0x7f6e2812b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44244/0 (socket says 172.21.15.155:44244) 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2d6c4700 1 -- 172.21.15.155:0/3347918625 learned_addr learned my addr 172.21.15.155:0/3347918625 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 -- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 msgr2=0x7f6e2812b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 --2- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 0x7f6e2812b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 -- 172.21.15.155:0/3347918625 >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 msgr2=0x7f6e2812ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 --2- 172.21.15.155:0/3347918625 >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 0x7f6e2812ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 -- 172.21.15.155:0/3347918625 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280989f0 con 0x7f6e2812b6e0 2024-04-25T11:02:54.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2cec3700 1 --2- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 0x7f6e2812b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2d6c4700 1 --2- 172.21.15.155:0/3347918625 >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 0x7f6e2812ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.197+0000 7f6e2dec5700 1 --2- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812bb70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009df0 tx=0x7f6e2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ed54b194f4a945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e1ffff700 1 -- 172.21.15.155:0/3347918625 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2000e070 con 0x7f6e2812b6e0 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e1ffff700 1 -- 172.21.15.155:0/3347918625 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2000baa0 con 0x7f6e2812b6e0 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e1ffff700 1 -- 172.21.15.155:0/3347918625 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2000be50 con 0x7f6e2812b6e0 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 -- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 msgr2=0x7f6e2812bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 --2- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812bb70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009df0 tx=0x7f6e2000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 -- 172.21.15.155:0/3347918625 shutdown_connections 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 --2- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812bb70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 --2- 172.21.15.155:0/3347918625 >> v2:172.21.15.155:3300/0 conn(0x7f6e2812e6a0 0x7f6e2812ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 --2- 172.21.15.155:0/3347918625 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812efc0 0x7f6e2812b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 -- 172.21.15.155:0/3347918625 >> 172.21.15.155:0/3347918625 conn(0x7f6e2809d830 msgr2=0x7f6e2809dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 -- 172.21.15.155:0/3347918625 shutdown_connections 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.198+0000 7f6e2f928700 1 -- 172.21.15.155:0/3347918625 wait complete. 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 Processor -- start 2024-04-25T11:02:54.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 -- start start 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812e6a0 0x7f6e281304e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e2812efc0 0x7f6e28130a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e281332a0 con 0x7f6e2812efc0 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28132f80 con 0x7f6e2812b6e0 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2f928700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e281315b0 con 0x7f6e2812e6a0 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2d6c4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.199+0000 7f6e2cec3700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812e6a0 0x7f6e281304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40496/0 (socket says 172.21.15.155:40496) 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2dec5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e2812efc0 0x7f6e28130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 -- 172.21.15.155:0/30535255 learned_addr learned my addr 172.21.15.155:0/30535255 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 -- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812e6a0 msgr2=0x7f6e281304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812e6a0 0x7f6e281304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 -- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e2812efc0 msgr2=0x7f6e28130a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e2812efc0 0x7f6e28130a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 -- 172.21.15.155:0/30535255 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18009550 con 0x7f6e2812b6e0 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2cec3700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812e6a0 0x7f6e281304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2dec5700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e2812efc0 0x7f6e28130a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2d6c4700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812ffa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ebb0 tx=0x7f6e1800eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18004030 con 0x7f6e2812b6e0 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18004190 con 0x7f6e2812b6e0 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e240094a0 con 0x7f6e2812b6e0 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18004520 con 0x7f6e2812b6e0 2024-04-25T11:02:54.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.200+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e20009a70 con 0x7f6e2812b6e0 2024-04-25T11:02:54.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.201+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f6e1800c850 con 0x7f6e2812b6e0 2024-04-25T11:02:54.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.201+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e28099680 con 0x7f6e2812b6e0 2024-04-25T11:02:54.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.202+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f6e1805c640 con 0x7f6e2812b6e0 2024-04-25T11:02:54.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.205+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f6e1803a360 con 0x7f6e2812b6e0 2024-04-25T11:02:54.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.430+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6e2809b1e0 con 0x7f6e2812b6e0 2024-04-25T11:02:54.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.431+0000 7f6e1dffb700 1 -- 172.21.15.155:0/30535255 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+447 (secure 0 0 0) 0x7f6e1803d980 con 0x7f6e2812b6e0 2024-04-25T11:02:54.431 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "mon": { 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph version 18.2.3 (76424b2fe1bb19c32c52140f39764599abf5e035) reef (stable)": 3 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "mgr": { 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph version 18.2.3 (76424b2fe1bb19c32c52140f39764599abf5e035) reef (stable)": 2 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": { 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph version 18.2.3 (76424b2fe1bb19c32c52140f39764599abf5e035) reef (stable)": 4 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "overall": { 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph version 18.2.3 (76424b2fe1bb19c32c52140f39764599abf5e035) reef (stable)": 9 2024-04-25T11:02:54.432 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.433 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.432+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 >> v1:172.21.15.155:6816/49624 conn(0x7f6e0c047560 legacy=0x7f6e0c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 msgr2=0x7f6e2812ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812ffa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ebb0 tx=0x7f6e1800eec0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 shutdown_connections 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e2812efc0 0x7f6e28130a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e2812b6e0 0x7f6e2812ffa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 --2- 172.21.15.155:0/30535255 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f6e2812e6a0 0x7f6e281304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 >> 172.21.15.155:0/30535255 conn(0x7f6e2809d830 msgr2=0x7f6e2809cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:54.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 shutdown_connections 2024-04-25T11:02:54.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.433+0000 7f6e2f928700 1 -- 172.21.15.155:0/30535255 wait complete. 2024-04-25T11:02:54.445 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2024-04-25T11:02:54.562 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.561+0000 7f399fce8700 1 Processor -- start 2024-04-25T11:02:54.562 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.561+0000 7f399fce8700 1 -- start start 2024-04-25T11:02:54.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399fce8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 0x7f399812cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399fce8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 0x7f399806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399fce8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 0x7f399806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399fce8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39980711e0 con 0x7f399812d180 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399fce8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399809ae20 con 0x7f39980705a0 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399fce8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f399812c660 con 0x7f399812c860 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399e285700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 0x7f399806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399d283700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 0x7f399806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 0x7f399812cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 0x7f399812cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44260/0 (socket says 172.21.15.155:44260) 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399e285700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 0x7f399806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40498/0 (socket says 172.21.15.155:40498) 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399d283700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 0x7f399806e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40440/0 (socket says 172.21.15.155:40440) 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 -- 172.21.15.155:0/1391594233 learned_addr learned my addr 172.21.15.155:0/1391594233 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 -- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 msgr2=0x7f399806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 --2- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 0x7f399806eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 -- 172.21.15.155:0/1391594233 >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 msgr2=0x7f399806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 --2- 172.21.15.155:0/1391594233 >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 0x7f399806e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 -- 172.21.15.155:0/1391594233 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998070d80 con 0x7f399812c860 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399d283700 1 --2- 172.21.15.155:0/1391594233 >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 0x7f399806e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399e285700 1 --2- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 0x7f399806eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399da84700 1 --2- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 0x7f399812cc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f39880098f0 tx=0x7f3988009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef6c145b83ac7cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.562+0000 7f399ca82700 1 -- 172.21.15.155:0/1391594233 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988011830 con 0x7f399812c860 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399ca82700 1 -- 172.21.15.155:0/1391594233 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3988011990 con 0x7f399812c860 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399ca82700 1 -- 172.21.15.155:0/1391594233 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988011d20 con 0x7f399812c860 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 -- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 msgr2=0x7f399812cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 --2- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 0x7f399812cc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f39880098f0 tx=0x7f3988009c00 comp rx=0 tx=0).stop 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 -- 172.21.15.155:0/1391594233 shutdown_connections 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 --2- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39980705a0 0x7f399806eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 --2- 172.21.15.155:0/1391594233 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812c860 0x7f399812cc40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 --2- 172.21.15.155:0/1391594233 >> v2:172.21.15.155:3300/0 conn(0x7f399812d180 0x7f399806e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 -- 172.21.15.155:0/1391594233 >> 172.21.15.155:0/1391594233 conn(0x7f399809bbc0 msgr2=0x7f3998099490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:54.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 -- 172.21.15.155:0/1391594233 shutdown_connections 2024-04-25T11:02:54.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.563+0000 7f399fce8700 1 -- 172.21.15.155:0/1391594233 wait complete. 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.564+0000 7f399fce8700 1 Processor -- start 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.564+0000 7f399fce8700 1 -- start start 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399fce8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39980705a0 0x7f39981bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399fce8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399812c860 0x7f39981be060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399fce8700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812d180 0x7f39981be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399fce8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980950f0 con 0x7f39980705a0 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399fce8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399812dd90 con 0x7f399812c860 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399fce8700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39981298b0 con 0x7f399812d180 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399da84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39980705a0 0x7f39981bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812d180 0x7f39981be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399d283700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399812c860 0x7f39981be060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399da84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39980705a0 0x7f39981bdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40450/0 (socket says 172.21.15.155:40450) 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399da84700 1 -- 172.21.15.155:0/3272548562 learned_addr learned my addr 172.21.15.155:0/3272548562 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 -- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399812c860 msgr2=0x7f39981be060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399812c860 0x7f39981be060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 -- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39980705a0 msgr2=0x7f39981bdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39980705a0 0x7f39981bdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 -- 172.21.15.155:0/3272548562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009a70 con 0x7f399812d180 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.565+0000 7f399e285700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812d180 0x7f39981be5a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f399000ef00 tx=0x7f399000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.566+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990010040 con 0x7f399812d180 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.566+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399000ba80 con 0x7f399812d180 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.566+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000be10 con 0x7f399812d180 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.566+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940094a0 con 0x7f399812d180 2024-04-25T11:02:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.566+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988009550 con 0x7f399812d180 2024-04-25T11:02:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.566+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f399001d410 con 0x7f399812d180 2024-04-25T11:02:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.567+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3998062510 con 0x7f399812d180 2024-04-25T11:02:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.567+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f399005c870 con 0x7f399812d180 2024-04-25T11:02:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.571+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f399003a360 con 0x7f399812d180 2024-04-25T11:02:54.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.731+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f399806fd90 con 0x7f399812d180 2024-04-25T11:02:54.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.732+0000 7f398e7fc700 1 -- 172.21.15.155:0/3272548562 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+16411 (secure 0 0 0) 0x7f3990018070 con 0x7f399812d180 2024-04-25T11:02:54.733 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:54.733 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-25T11:02:54.733 INFO:teuthology.orchestra.run.smithi155.stdout: "epoch": 26, 2024-04-25T11:02:54.733 INFO:teuthology.orchestra.run.smithi155.stdout: "fsid": "f023c376-0581-4e8c-aaca-f3f02914116c", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "created": "2024-04-25T11:00:50.077657+0000", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "modified": "2024-04-25T11:02:54.040550+0000", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "last_up_change": "2024-04-25T11:02:53.034844+0000", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "last_in_change": "2024-04-25T11:00:51.430090+0000", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "flags_num": 5799936, 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "flags_set": [ 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "pglog_hardlimit", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "purged_snapdirs", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_deletes", 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "sortbitwise" 2024-04-25T11:02:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: ], 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "crush_version": 9, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_snapshot": "", 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "pool_max": 2, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "max_osd": 4, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "require_min_compat_client": "pacific", 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "min_compat_client": "jewel", 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "require_osd_release": "pacific", 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "allow_crimson": false, 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "pools": [ 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.735 INFO:teuthology.orchestra.run.smithi155.stdout: "pool": 1, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "pool_name": ".mgr", 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "create_time": "2024-04-25T11:00:57.443026+0000", 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "flags": 1, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "flags_names": "hashpspool", 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "type": 1, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 2, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "min_size": 1, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "crush_rule": 0, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_count": 0, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_target": 0, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T11:02:54.736 INFO:teuthology.orchestra.run.smithi155.stdout: "object_hash": 2, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_autoscale_mode": "off", 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num": 1, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_placement_num": 1, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_placement_num_target": 1, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num_target": 1, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num_pending": 1, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "last_pg_merge_meta": { 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "source_pgid": "0.0", 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "ready_epoch": 0, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "last_epoch_started": 0, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "last_epoch_clean": 0, 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "source_version": "0'0", 2024-04-25T11:02:54.737 INFO:teuthology.orchestra.run.smithi155.stdout: "target_version": "0'0" 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "last_change": "17", 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "last_force_op_resend": "0", 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "auid": 0, 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "snap_mode": "selfmanaged", 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "snap_seq": 0, 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "snap_epoch": 0, 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "pool_snaps": [], 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "removed_snaps": "[]", 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "quota_max_bytes": 0, 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "quota_max_objects": 0, 2024-04-25T11:02:54.738 INFO:teuthology.orchestra.run.smithi155.stdout: "tiers": [], 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "tier_of": -1, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "read_tier": -1, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "write_tier": -1, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_mode": "none", 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "target_max_bytes": 0, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "target_max_objects": 0, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_min_flush_age": 0, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_min_evict_age": 0, 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "erasure_code_profile": "", 2024-04-25T11:02:54.739 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_params": { 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "none" 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_period": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_count": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "use_gmt_hitset": true, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "min_read_recency_for_promote": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "min_write_recency_for_promote": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_search_last_n": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "grade_table": [], 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "stripe_width": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "expected_num_objects": 0, 2024-04-25T11:02:54.740 INFO:teuthology.orchestra.run.smithi155.stdout: "fast_read": false, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "options": { 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num_max": 32, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num_min": 1 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "application_metadata": { 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "mgr": {}, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "mgr_devicehealth": {} 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "read_balance": { 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "score_acting": 4, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "score_stable": 4, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "optimal_score": 0.5, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "raw_score_acting": 2, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "raw_score_stable": 2, 2024-04-25T11:02:54.741 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_affinity_weighted": 1, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "average_primary_affinity": 1, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "average_primary_affinity_weighted": 1 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "pool": 2, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "pool_name": "rbd", 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "create_time": "2024-04-25T11:00:59.312325+0000", 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "flags": 1, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "flags_names": "hashpspool", 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "type": 1, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 2, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "min_size": 1, 2024-04-25T11:02:54.742 INFO:teuthology.orchestra.run.smithi155.stdout: "crush_rule": 0, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_count": 0, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_target": 0, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "object_hash": 2, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_autoscale_mode": "off", 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num": 8, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_placement_num": 8, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_placement_num_target": 8, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num_target": 8, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_num_pending": 8, 2024-04-25T11:02:54.743 INFO:teuthology.orchestra.run.smithi155.stdout: "last_pg_merge_meta": { 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "source_pgid": "0.0", 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "ready_epoch": 0, 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "last_epoch_started": 0, 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "last_epoch_clean": 0, 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "source_version": "0'0", 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "target_version": "0'0" 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "last_change": "13", 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "last_force_op_resend": "0", 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T11:02:54.744 INFO:teuthology.orchestra.run.smithi155.stdout: "auid": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "snap_mode": "selfmanaged", 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "snap_seq": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "snap_epoch": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "pool_snaps": [], 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "removed_snaps": "[]", 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "quota_max_bytes": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "quota_max_objects": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "tiers": [], 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "tier_of": -1, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "read_tier": -1, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "write_tier": -1, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_mode": "none", 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "target_max_bytes": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "target_max_objects": 0, 2024-04-25T11:02:54.745 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_min_flush_age": 0, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "cache_min_evict_age": 0, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "erasure_code_profile": "", 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_params": { 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "none" 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_period": 0, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_count": 0, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "use_gmt_hitset": true, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "min_read_recency_for_promote": 0, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "min_write_recency_for_promote": 0, 2024-04-25T11:02:54.746 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "hit_set_search_last_n": 0, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "grade_table": [], 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "stripe_width": 0, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "expected_num_objects": 0, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "fast_read": false, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "options": {}, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "application_metadata": { 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "rbd": {} 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "read_balance": { 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "score_acting": 2.5, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "score_stable": 2.5, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "optimal_score": 1, 2024-04-25T11:02:54.747 INFO:teuthology.orchestra.run.smithi155.stdout: "raw_score_acting": 2.5, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "raw_score_stable": 2.5, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_affinity_weighted": 1, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "average_primary_affinity": 1, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "average_primary_affinity_weighted": 1 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: ], 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "osds": [ 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 0, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "uuid": "a1a66333-2acf-451b-b7d0-2dfa2e55600f", 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "up": 1, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "in": 1, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "weight": 1, 2024-04-25T11:02:54.748 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_affinity": 1, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_begin": 9, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_end": 16, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "up_from": 22, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "up_thru": 22, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "down_at": 17, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "lost_at": 0, 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addrs": { 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6800", 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51324 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.749 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addrs": { 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6801", 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51324 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addrs": { 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6803", 2024-04-25T11:02:54.750 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51324 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addrs": { 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6802", 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51324 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addr": "172.21.15.155:6800/51324", 2024-04-25T11:02:54.751 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addr": "172.21.15.155:6801/51324", 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addr": "172.21.15.155:6803/51324", 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addr": "172.21.15.155:6802/51324", 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "state": [ 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "exists", 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "up" 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 1, 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "uuid": "7051096c-4e97-4ace-9814-455da6c8a06b", 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "up": 1, 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "in": 1, 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "weight": 1, 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_affinity": 1, 2024-04-25T11:02:54.752 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_begin": 10, 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_end": 17, 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "up_from": 23, 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "up_thru": 23, 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "down_at": 18, 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "lost_at": 0, 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addrs": { 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6808", 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51418 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.753 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addrs": { 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6809", 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51418 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addrs": { 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6811", 2024-04-25T11:02:54.754 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51418 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addrs": { 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6810", 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51418 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addr": "172.21.15.155:6808/51418", 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addr": "172.21.15.155:6809/51418", 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addr": "172.21.15.155:6811/51418", 2024-04-25T11:02:54.755 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addr": "172.21.15.155:6810/51418", 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "state": [ 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "exists", 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "up" 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 2, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "uuid": "faf1e6c0-e94a-4bf3-830d-67564885faaa", 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "up": 1, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "in": 1, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "weight": 1, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_affinity": 1, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_begin": 10, 2024-04-25T11:02:54.756 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_end": 17, 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "up_from": 24, 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "up_thru": 25, 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "down_at": 19, 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "lost_at": 0, 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addrs": { 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6804", 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51529 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.757 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addrs": { 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6805", 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51529 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addrs": { 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6807", 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51529 2024-04-25T11:02:54.758 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addrs": { 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6806", 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51529 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addr": "172.21.15.155:6804/51529", 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addr": "172.21.15.155:6805/51529", 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addr": "172.21.15.155:6807/51529", 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addr": "172.21.15.155:6806/51529", 2024-04-25T11:02:54.759 INFO:teuthology.orchestra.run.smithi155.stdout: "state": [ 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "exists", 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "up" 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 3, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "uuid": "4e269fdb-6854-450b-9956-6ab00516289e", 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "up": 1, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "in": 1, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "weight": 1, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_affinity": 1, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_begin": 9, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "last_clean_end": 19, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "up_from": 25, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "up_thru": 25, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "down_at": 20, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "lost_at": 0, 2024-04-25T11:02:54.760 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addrs": { 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6812", 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51677 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addrs": { 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6813", 2024-04-25T11:02:54.761 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51677 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addrs": { 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6815", 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51677 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addrs": { 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: "addrvec": [ 2024-04-25T11:02:54.762 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "172.21.15.155:6814", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 51677 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "public_addr": "172.21.15.155:6812/51677", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "cluster_addr": "172.21.15.155:6813/51677", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_back_addr": "172.21.15.155:6815/51677", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "heartbeat_front_addr": "172.21.15.155:6814/51677", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "state": [ 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "exists", 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: "up" 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-25T11:02:54.763 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: ], 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_xinfo": [ 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 0, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "down_stamp": "0.000000", 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_probability": 0, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_interval": 0, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "features": 4540138322906710015, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "old_weight": 0, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "last_purged_snaps_scrub": "2024-04-25T11:00:56.389357+0000", 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "dead_epoch": 17 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 1, 2024-04-25T11:02:54.764 INFO:teuthology.orchestra.run.smithi155.stdout: "down_stamp": "0.000000", 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_probability": 0, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_interval": 0, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "features": 4540138322906710015, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "old_weight": 0, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "last_purged_snaps_scrub": "2024-04-25T11:00:56.579751+0000", 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "dead_epoch": 18 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 2, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "down_stamp": "0.000000", 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_probability": 0, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_interval": 0, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "features": 4540138322906710015, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "old_weight": 0, 2024-04-25T11:02:54.765 INFO:teuthology.orchestra.run.smithi155.stdout: "last_purged_snaps_scrub": "2024-04-25T11:00:56.625639+0000", 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "dead_epoch": 19 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "osd": 3, 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "down_stamp": "2024-04-25T11:02:47.002020+0000", 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_probability": 0, 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "laggy_interval": 0, 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "features": 4540138322906710015, 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "old_weight": 0, 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "last_purged_snaps_scrub": "2024-04-25T11:00:56.361390+0000", 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "dead_epoch": 0 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: ], 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_upmap": [], 2024-04-25T11:02:54.766 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_upmap_items": [], 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_upmap_primaries": [], 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "pg_temp": [], 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "primary_temp": [], 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklist": { 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "172.21.15.155:0/3077341019": "2024-04-26T11:02:39.971273+0000", 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "172.21.15.155:0/966131479": "2024-04-26T11:02:39.971273+0000", 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "172.21.15.155:6816/43298": "2024-04-26T11:02:39.971273+0000" 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "range_blocklist": {}, 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "erasure_code_profiles": { 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "default": { 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "crush-failure-domain": "osd", 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "k": "2", 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "m": "1", 2024-04-25T11:02:54.767 INFO:teuthology.orchestra.run.smithi155.stdout: "plugin": "jerasure", 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "technique": "reed_sol_van" 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "removed_snaps_queue": [], 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "new_removed_snaps": [], 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "new_purged_snaps": [], 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "crush_node_flags": {}, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "device_class_flags": {}, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "stretch_mode": { 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "stretch_mode_enabled": false, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "stretch_bucket_count": 0, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "degraded_stretch_mode": 0, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "recovering_stretch_mode": 0, 2024-04-25T11:02:54.768 INFO:teuthology.orchestra.run.smithi155.stdout: "stretch_mode_bucket": 0 2024-04-25T11:02:54.769 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-25T11:02:54.769 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-25T11:02:54.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 >> v1:172.21.15.155:6816/49624 conn(0x7f397c047560 legacy=0x7f397c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812d180 msgr2=0x7f39981be5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812d180 0x7f39981be5a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f399000ef00 tx=0x7f399000b390 comp rx=0 tx=0).stop 2024-04-25T11:02:54.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 shutdown_connections 2024-04-25T11:02:54.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f399812d180 0x7f39981be5a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39980705a0 0x7f39981bdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 --2- 172.21.15.155:0/3272548562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399812c860 0x7f39981be060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 >> 172.21.15.155:0/3272548562 conn(0x7f399809bbc0 msgr2=0x7f39980984b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:54.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 shutdown_connections 2024-04-25T11:02:54.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.734+0000 7f399fce8700 1 -- 172.21.15.155:0/3272548562 wait complete. 2024-04-25T11:02:54.771 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-04-25T11:02:54.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.887+0000 7f28053fd700 1 Processor -- start 2024-04-25T11:02:54.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.887+0000 7f28053fd700 1 -- start start 2024-04-25T11:02:54.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f28053fd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f280012f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f28053fd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 0x7f280006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f28053fd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f280006ccd0 0x7f280006d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f28053fd700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280006ac20 con 0x7f280006ccd0 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f28053fd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280006f8f0 con 0x7f280012f130 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f28053fd700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f280006f1c0 con 0x7f280012fa50 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f280006ccd0 0x7f280006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27fe7fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 0x7f280006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27feffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f280012f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f280006ccd0 0x7f280006d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40456/0 (socket says 172.21.15.155:40456) 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27feffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f280012f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40520/0 (socket says 172.21.15.155:40520) 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 -- 172.21.15.155:0/2209273331 learned_addr learned my addr 172.21.15.155:0/2209273331 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27fe7fc700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 0x7f280006c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44268/0 (socket says 172.21.15.155:44268) 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 -- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 msgr2=0x7f280006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 0x7f280006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 -- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 msgr2=0x7f280012f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f280012f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 -- 172.21.15.155:0/2209273331 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280006f330 con 0x7f280006ccd0 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27fe7fc700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 0x7f280006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27feffd700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f280012f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.888+0000 7f27ff7fe700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280006ccd0 0x7f280006d180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009e50 tx=0x7f27f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eceff6c948c66d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f27fdffb700 1 -- 172.21.15.155:0/2209273331 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800e070 con 0x7f280006ccd0 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f27fdffb700 1 -- 172.21.15.155:0/2209273331 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f800b910 con 0x7f280006ccd0 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f27fdffb700 1 -- 172.21.15.155:0/2209273331 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800bcc0 con 0x7f280006ccd0 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 -- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280006ccd0 msgr2=0x7f280006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280006ccd0 0x7f280006d180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009e50 tx=0x7f27f800b330 comp rx=0 tx=0).stop 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 -- 172.21.15.155:0/2209273331 shutdown_connections 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280006ccd0 0x7f280006d180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f280012f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 --2- 172.21.15.155:0/2209273331 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280012fa50 0x7f280006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 -- 172.21.15.155:0/2209273331 >> 172.21.15.155:0/2209273331 conn(0x7f280009ef90 msgr2=0x7f280009f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:54.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 -- 172.21.15.155:0/2209273331 shutdown_connections 2024-04-25T11:02:54.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.889+0000 7f28053fd700 1 -- 172.21.15.155:0/2209273331 wait complete. 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.890+0000 7f28053fd700 1 Processor -- start 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.890+0000 7f28053fd700 1 -- start start 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 0x7f28001c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f28001c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280012fa50 0x7f28001c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000966d0 con 0x7f280012fa50 2024-04-25T11:02:54.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2800094da0 con 0x7f280012f130 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28000956b0 con 0x7f280006ccd0 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 0x7f28001c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27fe7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f28001c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 0x7f28001c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44284/0 (socket says 172.21.15.155:44284) 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 -- 172.21.15.155:0/3239854420 learned_addr learned my addr 172.21.15.155:0/3239854420 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27ff7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280012fa50 0x7f28001c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 -- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 msgr2=0x7f28001c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f28001c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 -- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280012fa50 msgr2=0x7f28001c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280012fa50 0x7f28001c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 -- 172.21.15.155:0/3239854420 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f40094a0 con 0x7f280006ccd0 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27ff7fe700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280012fa50 0x7f28001c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27feffd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 0x7f28001c3180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27f400ebe0 tx=0x7f27f400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:54.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4009780 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f400c850 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f00094a0 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f400cbe0 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.891+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f8009b30 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.892+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2800067390 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.892+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f27f400cd40 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.893+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f27f405c7b0 con 0x7f280006ccd0 2024-04-25T11:02:54.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:54.896+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f27f403a4d0 con 0x7f280006ccd0 2024-04-25T11:02:55.108 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.108+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2800062510 con 0x7f280006ccd0 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.108+0000 7f27e77fe700 1 -- 172.21.15.155:0/3239854420 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7f27f4017070 con 0x7f280006ccd0 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stdout:e1 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stdout:legacy client fscid: -1 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:55.109 INFO:teuthology.orchestra.run.smithi155.stdout:No filesystems configured 2024-04-25T11:02:55.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 >> v1:172.21.15.155:6816/49624 conn(0x7f27ec047560 legacy=0x7f27ec049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:55.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 msgr2=0x7f28001c3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:55.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 0x7f28001c3180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27f400ebe0 tx=0x7f27f400eef0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 shutdown_connections 2024-04-25T11:02:55.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280012fa50 0x7f28001c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f280006ccd0 0x7f28001c3180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 --2- 172.21.15.155:0/3239854420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f280012f130 0x7f28001c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 >> 172.21.15.155:0/3239854420 conn(0x7f280009ef90 msgr2=0x7f280012d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:55.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 shutdown_connections 2024-04-25T11:02:55.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.110+0000 7f28053fd700 1 -- 172.21.15.155:0/3239854420 wait complete. 2024-04-25T11:02:55.112 INFO:teuthology.orchestra.run.smithi155.stderr:dumped fsmap epoch 1 2024-04-25T11:02:55.123 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release quincy' 2024-04-25T11:02:55.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.250+0000 7febc8cca700 1 Processor -- start 2024-04-25T11:02:55.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.250+0000 7febc8cca700 1 -- start start 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc8cca700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4097c70 0x7febc4131df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc8cca700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7febc4132330 0x7febc4134720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc8cca700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc40983e0 con 0x7febc4097180 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc8cca700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc40684a0 con 0x7febc4132330 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc8cca700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc409c0f0 con 0x7febc4097c70 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc2d9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7febc4132330 0x7febc4134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc1d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4097c70 0x7febc4131df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc259c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7febc4097180 0x7febc4097560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35294/0 (socket says 172.21.15.155:35294) 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc259c700 1 -- 172.21.15.155:0/3759081320 learned_addr learned my addr 172.21.15.155:0/3759081320 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc1d9b700 1 -- 172.21.15.155:0/3759081320 >> v2:172.21.15.155:3300/0 conn(0x7febc4132330 msgr2=0x7febc4134720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc1d9b700 1 --2- 172.21.15.155:0/3759081320 >> v2:172.21.15.155:3300/0 conn(0x7febc4132330 0x7febc4134720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc1d9b700 1 -- 172.21.15.155:0/3759081320 >> v1:172.21.15.155:6789/0 conn(0x7febc4097180 legacy=0x7febc4097560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:55.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc1d9b700 1 -- 172.21.15.155:0/3759081320 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc4098cb0 con 0x7febc4097c70 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc2d9d700 1 --2- 172.21.15.155:0/3759081320 >> v2:172.21.15.155:3300/0 conn(0x7febc4132330 0x7febc4134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc1d9b700 1 --2- 172.21.15.155:0/3759081320 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4097c70 0x7febc4131df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febb8009840 tx=0x7febb8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c64bb5e40ffe398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc159a700 1 -- 172.21.15.155:0/3759081320 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8011830 con 0x7febc4097c70 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc159a700 1 -- 172.21.15.155:0/3759081320 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8011990 con 0x7febc4097c70 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.251+0000 7febc159a700 1 -- 172.21.15.155:0/3759081320 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8011d20 con 0x7febc4097c70 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 -- 172.21.15.155:0/3759081320 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4097c70 msgr2=0x7febc4131df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 --2- 172.21.15.155:0/3759081320 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4097c70 0x7febc4131df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febb8009840 tx=0x7febb8009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 -- 172.21.15.155:0/3759081320 shutdown_connections 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 --2- 172.21.15.155:0/3759081320 >> v2:172.21.15.155:3300/0 conn(0x7febc4132330 0x7febc4134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 --2- 172.21.15.155:0/3759081320 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4097c70 0x7febc4131df0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 -- 172.21.15.155:0/3759081320 >> 172.21.15.155:0/3759081320 conn(0x7febc409f2a0 msgr2=0x7febc409b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 -- 172.21.15.155:0/3759081320 shutdown_connections 2024-04-25T11:02:55.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.252+0000 7febc8cca700 1 -- 172.21.15.155:0/3759081320 wait complete. 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 Processor -- start 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 -- start start 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc4097180 0x7febc4130280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 0x7febc41307c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4132330 0x7febc4130d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc406dc20 con 0x7febc4097180 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc406cd90 con 0x7febc4097c70 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.253+0000 7febc8cca700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc4131ca0 con 0x7febc4132330 2024-04-25T11:02:55.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc1d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 0x7febc41307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc4097180 0x7febc4130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc1d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 0x7febc41307c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40560/0 (socket says 172.21.15.155:40560) 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc1d9b700 1 -- 172.21.15.155:0/2003648437 learned_addr learned my addr 172.21.15.155:0/2003648437 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4132330 0x7febc4130d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 -- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 msgr2=0x7febc41307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 0x7febc41307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 -- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc4097180 msgr2=0x7febc4130280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc4097180 0x7febc4130280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 -- 172.21.15.155:0/2003648437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc009a70 con 0x7febc4132330 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc1d9b700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 0x7febc41307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc2d9d700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4132330 0x7febc4130d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ef90 tx=0x7febbc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00cc90 con 0x7febc4132330 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc00f040 con 0x7febc4132330 2024-04-25T11:02:55.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb80094a0 con 0x7febc4132330 2024-04-25T11:02:55.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0146c0 con 0x7febc4132330 2024-04-25T11:02:55.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.254+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc406d2c0 con 0x7febc4132330 2024-04-25T11:02:55.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.255+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7febbc014820 con 0x7febc4132330 2024-04-25T11:02:55.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.255+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc4068040 con 0x7febc4132330 2024-04-25T11:02:55.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.255+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2961+0+0 (secure 0 0 0) 0x7febbc05d250 con 0x7febc4132330 2024-04-25T11:02:55.260 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.259+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7febbc03ada0 con 0x7febc4132330 2024-04-25T11:02:55.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:55.443+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd require-osd-release", "release": "quincy"} v 0) v1 -- 0x7febc4062510 con 0x7febc4132330 2024-04-25T11:02:56.215 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.214+0000 7febb2ffd700 1 -- 172.21.15.155:0/2003648437 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "quincy"}]=0 v27) v1 ==== 92+0+0 (secure 0 0 0) 0x7febbc018020 con 0x7febc4132330 2024-04-25T11:02:56.218 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 >> v1:172.21.15.155:6816/49624 conn(0x7feba4047590 legacy=0x7feba4049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4132330 msgr2=0x7febc4130d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4132330 0x7febc4130d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ef90 tx=0x7febbc00c3a0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 shutdown_connections 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7febc4132330 0x7febc4130d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc4097180 0x7febc4130280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 --2- 172.21.15.155:0/2003648437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4097c70 0x7febc41307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 >> 172.21.15.155:0/2003648437 conn(0x7febc409f2a0 msgr2=0x7febc4099620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 shutdown_connections 2024-04-25T11:02:56.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.218+0000 7febc8cca700 1 -- 172.21.15.155:0/2003648437 wait complete. 2024-04-25T11:02:56.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 Processor -- start 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 -- start start 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8d4006d7a0 0x7f8d4006db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d4012ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 0x7f8d4012f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4006bdd0 con 0x7f8d4006d7a0 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40072830 con 0x7f8d4012f140 2024-04-25T11:02:56.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.351+0000 7f8d449af700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40072470 con 0x7f8d4006e0c0 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3dd9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d4012ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8d4006d7a0 0x7f8d4006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3ed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 0x7f8d4012f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8d4006d7a0 0x7f8d4006db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40514/0 (socket says 172.21.15.155:40514) 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3dd9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d4012ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44342/0 (socket says 172.21.15.155:44342) 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3ed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 0x7f8d4012f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40576/0 (socket says 172.21.15.155:40576) 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 -- 172.21.15.155:0/3662926983 learned_addr learned my addr 172.21.15.155:0/3662926983 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 -- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 msgr2=0x7f8d4012ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d4012ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 -- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 msgr2=0x7f8d4012f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 0x7f8d4012f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 -- 172.21.15.155:0/3662926983 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4006c3f0 con 0x7f8d4006d7a0 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3dd9b700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d4012ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3ed9d700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 0x7f8d4012f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3e59c700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4006d7a0 0x7f8d4006db80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d300098f0 tx=0x7f8d30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c2fd504600e8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3d59a700 1 -- 172.21.15.155:0/3662926983 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30011830 con 0x7f8d4006d7a0 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3d59a700 1 -- 172.21.15.155:0/3662926983 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d30011990 con 0x7f8d4006d7a0 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.352+0000 7f8d3d59a700 1 -- 172.21.15.155:0/3662926983 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30011d20 con 0x7f8d4006d7a0 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 -- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4006d7a0 msgr2=0x7f8d4006db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4006d7a0 0x7f8d4006db80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d300098f0 tx=0x7f8d30009c00 comp rx=0 tx=0).stop 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 -- 172.21.15.155:0/3662926983 shutdown_connections 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4012f140 0x7f8d4012f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4006d7a0 0x7f8d4006db80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 --2- 172.21.15.155:0/3662926983 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d4012ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 -- 172.21.15.155:0/3662926983 >> 172.21.15.155:0/3662926983 conn(0x7f8d4009a760 msgr2=0x7f8d4009ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:56.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 -- 172.21.15.155:0/3662926983 shutdown_connections 2024-04-25T11:02:56.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.353+0000 7f8d449af700 1 -- 172.21.15.155:0/3662926983 wait complete. 2024-04-25T11:02:56.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 Processor -- start 2024-04-25T11:02:56.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 -- start start 2024-04-25T11:02:56.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 0x7f8d40094370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d400948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4012f140 0x7f8d40094df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40096780 con 0x7f8d4012f140 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401b6910 con 0x7f8d4006d7a0 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d449af700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40096fb0 con 0x7f8d4006e0c0 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d3e59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 0x7f8d40094370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d3dd9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d400948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d3ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4012f140 0x7f8d40094df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d3e59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 0x7f8d40094370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40590/0 (socket says 172.21.15.155:40590) 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d3dd9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d400948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44356/0 (socket says 172.21.15.155:44356) 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.354+0000 7f8d3e59c700 1 -- 172.21.15.155:0/1787814850 learned_addr learned my addr 172.21.15.155:0/1787814850 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3e59c700 1 -- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 msgr2=0x7f8d400948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3e59c700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d400948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3e59c700 1 -- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4012f140 msgr2=0x7f8d40094df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3e59c700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4012f140 0x7f8d40094df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3e59c700 1 -- 172.21.15.155:0/1787814850 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340094a0 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3ed9d700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4012f140 0x7f8d40094df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d3e59c700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 0x7f8d40094370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000c00 tx=0x7f8d30004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004240 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3001f070 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d38009a70 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300045c0 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.355+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d30009550 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.356+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40062510 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.356+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f8d30021030 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.356+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f8d3005d830 con 0x7f8d4006d7a0 2024-04-25T11:02:56.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.360+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f8d3003ecb0 con 0x7f8d4006d7a0 2024-04-25T11:02:56.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.518+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f8d4006a970 con 0x7f8d4006d7a0 2024-04-25T11:02:56.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.519+0000 7f8d2effd700 1 -- 172.21.15.155:0/1787814850 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v27) v1 ==== 59+0+9 (secure 0 0 0) 0x7f8d30015020 con 0x7f8d4006d7a0 2024-04-25T11:02:56.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.520+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 >> v1:172.21.15.155:6816/49624 conn(0x7f8d20047590 legacy=0x7f8d20049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.520+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 msgr2=0x7f8d40094370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.520+0000 7f8d449af700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 0x7f8d40094370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000c00 tx=0x7f8d30004750 comp rx=0 tx=0).stop 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.520+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 shutdown_connections 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.521+0000 7f8d449af700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4012f140 0x7f8d40094df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.521+0000 7f8d449af700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d4006d7a0 0x7f8d40094370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.521+0000 7f8d449af700 1 --2- 172.21.15.155:0/1787814850 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8d4006e0c0 0x7f8d400948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.521+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 >> 172.21.15.155:0/1787814850 conn(0x7f8d4009a760 msgr2=0x7f8d401b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.521+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 shutdown_connections 2024-04-25T11:02:56.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.521+0000 7f8d449af700 1 -- 172.21.15.155:0/1787814850 wait complete. 2024-04-25T11:02:56.630 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 Processor -- start 2024-04-25T11:02:56.630 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 -- start start 2024-04-25T11:02:56.630 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd0412ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.630 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd0412f3a0 0x7fdd0406b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.630 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0406bbc0 0x7fdd0406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.630 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0406c540 con 0x7fdd0412f3a0 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0412e860 con 0x7fdd0412ea80 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.627+0000 7fdd0b81e700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04095160 con 0x7fdd0406bbc0 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd095ba700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd0412ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd0412f3a0 0x7fdd0406b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd09dbb700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0406bbc0 0x7fdd0406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd0412f3a0 0x7fdd0406b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40526/0 (socket says 172.21.15.155:40526) 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd095ba700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd0412ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40596/0 (socket says 172.21.15.155:40596) 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 -- 172.21.15.155:0/789745986 learned_addr learned my addr 172.21.15.155:0/789745986 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd09dbb700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0406bbc0 0x7fdd0406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44372/0 (socket says 172.21.15.155:44372) 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 -- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0406bbc0 msgr2=0x7fdd0406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0406bbc0 0x7fdd0406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 -- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 msgr2=0x7fdd0412ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd0412ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 -- 172.21.15.155:0/789745986 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04099ef0 con 0x7fdd0412f3a0 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd095ba700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd0412ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdd08db9700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0412f3a0 0x7fdd0406b680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd000098f0 tx=0x7fdd00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fd513c45ccd0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdcfbfff700 1 -- 172.21.15.155:0/789745986 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00011830 con 0x7fdd0412f3a0 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdcfbfff700 1 -- 172.21.15.155:0/789745986 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd00011990 con 0x7fdd0412f3a0 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.628+0000 7fdcfbfff700 1 -- 172.21.15.155:0/789745986 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00011d20 con 0x7fdd0412f3a0 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 -- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0412f3a0 msgr2=0x7fdd0406b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0412f3a0 0x7fdd0406b680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd000098f0 tx=0x7fdd00009c00 comp rx=0 tx=0).stop 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 -- 172.21.15.155:0/789745986 shutdown_connections 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0406bbc0 0x7fdd0406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd0412ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/789745986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0412f3a0 0x7fdd0406b680 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 -- 172.21.15.155:0/789745986 >> 172.21.15.155:0/789745986 conn(0x7fdd0409ad20 msgr2=0x7fdd040985f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 -- 172.21.15.155:0/789745986 shutdown_connections 2024-04-25T11:02:56.632 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.629+0000 7fdd0b81e700 1 -- 172.21.15.155:0/789745986 wait complete. 2024-04-25T11:02:56.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 Processor -- start 2024-04-25T11:02:56.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 -- start start 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0406bbc0 0x7fdd0412feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd041303f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0412f3a0 0x7fdd041c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04133370 con 0x7fdd0406bbc0 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04130cf0 con 0x7fdd0412ea80 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd0b81e700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04130930 con 0x7fdd0412f3a0 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd08db9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd041303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd09dbb700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0412f3a0 0x7fdd041c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd08db9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd041303f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40606/0 (socket says 172.21.15.155:40606) 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd08db9700 1 -- 172.21.15.155:0/1390948745 learned_addr learned my addr 172.21.15.155:0/1390948745 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd095ba700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0406bbc0 0x7fdd0412feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd09dbb700 1 -- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 msgr2=0x7fdd041303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd09dbb700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd041303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd09dbb700 1 -- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0406bbc0 msgr2=0x7fdd0412feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.630+0000 7fdd09dbb700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0406bbc0 0x7fdd0412feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdd09dbb700 1 -- 172.21.15.155:0/1390948745 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40094a0 con 0x7fdd0412f3a0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdd08db9700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd041303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdd09dbb700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0412f3a0 0x7fdd041c2f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00eed0 tx=0x7fdcfc00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc010040 con 0x7fdd0412f3a0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc00ba80 con 0x7fdd0412f3a0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd00009550 con 0x7fdd0412f3a0 2024-04-25T11:02:56.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00be30 con 0x7fdd0412f3a0 2024-04-25T11:02:56.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.631+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd04132730 con 0x7fdd0412f3a0 2024-04-25T11:02:56.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.632+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd04062510 con 0x7fdd0412f3a0 2024-04-25T11:02:56.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.632+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 8) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fdcfc01d400 con 0x7fdd0412f3a0 2024-04-25T11:02:56.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.632+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fdcfc05cd10 con 0x7fdd0412f3a0 2024-04-25T11:02:56.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.636+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fdcfc03a9d0 con 0x7fdd0412f3a0 2024-04-25T11:02:56.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:56.800+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7fdd0406ac40 con 0x7fdd0412f3a0 2024-04-25T11:02:57.063 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.063+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fdcfc05b9b0 con 0x7fdd0412f3a0 2024-04-25T11:02:57.227 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.226+0000 7fdcf9ffb700 1 -- 172.21.15.155:0/1390948745 <== mon.2 v2:172.21.15.155:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fdcfc059ef0 con 0x7fdd0412f3a0 2024-04-25T11:02:57.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 >> v1:172.21.15.155:6816/49624 conn(0x7fdce8047560 legacy=0x7fdce8049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0412f3a0 msgr2=0x7fdd041c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0412f3a0 0x7fdd041c2f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00eed0 tx=0x7fdcfc00b390 comp rx=0 tx=0).stop 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 shutdown_connections 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fdd0412f3a0 0x7fdd041c2f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0406bbc0 0x7fdd0412feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 --2- 172.21.15.155:0/1390948745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0412ea80 0x7fdd041303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 >> 172.21.15.155:0/1390948745 conn(0x7fdd0409ad20 msgr2=0x7fdd04095cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 shutdown_connections 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.229+0000 7fdd0b81e700 1 -- 172.21.15.155:0/1390948745 wait complete. 2024-04-25T11:02:57.230 INFO:teuthology.orchestra.run.smithi155.stderr:set pool 1 pg_autoscale_mode to off 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 Processor -- start 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 -- start start 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7348098730 0x7f7348098b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f734812e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f734812f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7348099800 con 0x7f7348098730 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734806bd00 con 0x7f7348099050 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.336+0000 7f734dd53700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f734812f620 con 0x7f734812ec70 2024-04-25T11:02:57.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7348098730 0x7f7348098b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f7347fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f734812f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f7346ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f734812e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7348098730 0x7f7348098b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40546/0 (socket says 172.21.15.155:40546) 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 -- 172.21.15.155:0/401911297 learned_addr learned my addr 172.21.15.155:0/401911297 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 -- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 msgr2=0x7f734812f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f734812f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 -- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 msgr2=0x7f734812e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f734812e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 -- 172.21.15.155:0/401911297 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734806bf50 con 0x7f7348098730 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f7347fff700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f734812f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73477fe700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348098b10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73380098f0 tx=0x7f7338009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba932a3f3d5f0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73467fc700 1 -- 172.21.15.155:0/401911297 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338011830 con 0x7f7348098730 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73467fc700 1 -- 172.21.15.155:0/401911297 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338011990 con 0x7f7348098730 2024-04-25T11:02:57.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.337+0000 7f73467fc700 1 -- 172.21.15.155:0/401911297 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338011d20 con 0x7f7348098730 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 -- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 msgr2=0x7f7348098b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348098b10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73380098f0 tx=0x7f7338009c00 comp rx=0 tx=0).stop 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 -- 172.21.15.155:0/401911297 shutdown_connections 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f734812f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348098b10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 --2- 172.21.15.155:0/401911297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f734812e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 -- 172.21.15.155:0/401911297 >> 172.21.15.155:0/401911297 conn(0x7f73480a10f0 msgr2=0x7f73480a1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 -- 172.21.15.155:0/401911297 shutdown_connections 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.338+0000 7f734dd53700 1 -- 172.21.15.155:0/401911297 wait complete. 2024-04-25T11:02:57.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 Processor -- start 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 -- start start 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f73481332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f7348133800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348126c40 con 0x7f7348098730 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7348134fb0 con 0x7f7348099050 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f734dd53700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7348133e30 con 0x7f734812ec70 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f73477fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f73477fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40560/0 (socket says 172.21.15.155:40560) 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f73477fe700 1 -- 172.21.15.155:0/564918751 learned_addr learned my addr 172.21.15.155:0/564918751 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f7347fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f7348133800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.339+0000 7f7346ffd700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f73481332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:57.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f73477fe700 1 -- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 msgr2=0x7f7348133800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f73477fe700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f7348133800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f73477fe700 1 -- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 msgr2=0x7f73481332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f73477fe700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f73481332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f73477fe700 1 -- 172.21.15.155:0/564918751 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7340009a70 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f7347fff700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f7348133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f73477fe700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348132d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7338009f50 tx=0x7f7338004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338004450 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73380045b0 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7338009550 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380038a0 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.340+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73481c3140 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.341+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7348062510 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.341+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f7338021070 con 0x7f7348098730 2024-04-25T11:02:57.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.341+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f733805d9b0 con 0x7f7348098730 2024-04-25T11:02:57.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.345+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f7338026ad0 con 0x7f7348098730 2024-04-25T11:02:57.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:57.506+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f73481262e0 con 0x7f7348098730 2024-04-25T11:02:58.234 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.233+0000 7f732ffff700 1 -- 172.21.15.155:0/564918751 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v29) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7338015020 con 0x7f7348098730 2024-04-25T11:02:58.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 >> v1:172.21.15.155:6816/49624 conn(0x7f73340475f0 legacy=0x7f7334049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 msgr2=0x7f7348132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348132d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7338009f50 tx=0x7f7338004750 comp rx=0 tx=0).stop 2024-04-25T11:02:58.236 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 shutdown_connections 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f734812ec70 0x7f7348133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7348098730 0x7f7348132d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 --2- 172.21.15.155:0/564918751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7348099050 0x7f73481332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 >> 172.21.15.155:0/564918751 conn(0x7f73480a10f0 msgr2=0x7f734806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 shutdown_connections 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.235+0000 7f734dd53700 1 -- 172.21.15.155:0/564918751 wait complete. 2024-04-25T11:02:58.237 INFO:teuthology.orchestra.run.smithi155.stderr:set pool 2 pg_autoscale_mode to off 2024-04-25T11:02:58.248 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-04-25T11:02:58.258 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:02:58.259 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:02:58.259 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:02:58.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.378+0000 7fbfbbd57700 1 Processor -- start 2024-04-25T11:02:58.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.378+0000 7fbfbbd57700 1 -- start start 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfbbd57700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbfb412ebd0 0x7fbfb412efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfbbd57700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb406eba0 0x7fbfb406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfbbd57700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb40726b0 con 0x7fbfb412f4f0 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfbbd57700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb406f5b0 con 0x7fbfb412ebd0 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfbbd57700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb406c6e0 con 0x7fbfb406eba0 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfb9af3700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbfb412ebd0 0x7fbfb412efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfba2f4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb406eba0 0x7fbfb406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfb92f2700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fbfb412f4f0 0x7fbfb406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35310/0 (socket says 172.21.15.155:35310) 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfb92f2700 1 -- 172.21.15.155:0/3055524917 learned_addr learned my addr 172.21.15.155:0/3055524917 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfba2f4700 1 -- 172.21.15.155:0/3055524917 >> v2:172.21.15.155:3300/0 conn(0x7fbfb412ebd0 msgr2=0x7fbfb412efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfba2f4700 1 --2- 172.21.15.155:0/3055524917 >> v2:172.21.15.155:3300/0 conn(0x7fbfb412ebd0 0x7fbfb412efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfba2f4700 1 -- 172.21.15.155:0/3055524917 >> v1:172.21.15.155:6789/0 conn(0x7fbfb412f4f0 legacy=0x7fbfb406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfba2f4700 1 -- 172.21.15.155:0/3055524917 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb406fb70 con 0x7fbfb406eba0 2024-04-25T11:02:58.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfb9af3700 1 --2- 172.21.15.155:0/3055524917 >> v2:172.21.15.155:3300/0 conn(0x7fbfb412ebd0 0x7fbfb412efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.379+0000 7fbfba2f4700 1 --2- 172.21.15.155:0/3055524917 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb406eba0 0x7fbfb406eff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009d90 tx=0x7fbfac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c933a4ed9a28e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfb8af1700 1 -- 172.21.15.155:0/3055524917 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00e070 con 0x7fbfb406eba0 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfb8af1700 1 -- 172.21.15.155:0/3055524917 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00bb20 con 0x7fbfb406eba0 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfb8af1700 1 -- 172.21.15.155:0/3055524917 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac012430 con 0x7fbfb406eba0 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 -- 172.21.15.155:0/3055524917 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb406eba0 msgr2=0x7fbfb406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/3055524917 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb406eba0 0x7fbfb406eff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009d90 tx=0x7fbfac00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 -- 172.21.15.155:0/3055524917 shutdown_connections 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/3055524917 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb406eba0 0x7fbfb406eff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/3055524917 >> v2:172.21.15.155:3300/0 conn(0x7fbfb412ebd0 0x7fbfb412efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 -- 172.21.15.155:0/3055524917 >> 172.21.15.155:0/3055524917 conn(0x7fbfb409dbc0 msgr2=0x7fbfb409dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 -- 172.21.15.155:0/3055524917 shutdown_connections 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.380+0000 7fbfbbd57700 1 -- 172.21.15.155:0/3055524917 wait complete. 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 Processor -- start 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 -- start start 2024-04-25T11:02:58.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfb406eba0 0x7fbfb41c0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 0x7fbfb41c0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb412f4f0 0x7fbfb41c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4128e10 con 0x7fbfb412ebd0 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4129670 con 0x7fbfb406eba0 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfbbd57700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4127350 con 0x7fbfb412f4f0 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfb92f2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 0x7fbfb41c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfb9af3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfb406eba0 0x7fbfb41c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfba2f4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb412f4f0 0x7fbfb41c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfb92f2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 0x7fbfb41c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40570/0 (socket says 172.21.15.155:40570) 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfb9af3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfb406eba0 0x7fbfb41c0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40622/0 (socket says 172.21.15.155:40622) 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.381+0000 7fbfb92f2700 1 -- 172.21.15.155:0/2256269437 learned_addr learned my addr 172.21.15.155:0/2256269437 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfba2f4700 1 -- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfb406eba0 msgr2=0x7fbfb41c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfba2f4700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfb406eba0 0x7fbfb41c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfba2f4700 1 -- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 msgr2=0x7fbfb41c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfba2f4700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 0x7fbfb41c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfba2f4700 1 -- 172.21.15.155:0/2256269437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40094a0 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfb92f2700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 0x7fbfb41c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfba2f4700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb412f4f0 0x7fbfb41c32e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbfac016000 tx=0x7fbfac004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac012430 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac020070 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004280 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac009a70 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.382+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb41c0ad0 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.383+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4062510 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.383+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fbfac0043e0 con 0x7fbfb412f4f0 2024-04-25T11:02:58.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.383+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fbfac022070 con 0x7fbfb412f4f0 2024-04-25T11:02:58.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.387+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fbfac029080 con 0x7fbfb412f4f0 2024-04-25T11:02:58.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.583+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbfb409b110 con 0x7fbfb412f4f0 2024-04-25T11:02:58.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.585+0000 7fbfaa7fc700 1 -- 172.21.15.155:0/2256269437 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v9) v1 ==== 74+0+107686 (secure 0 0 0) 0x7fbfac012e60 con 0x7fbfb412f4f0 2024-04-25T11:02:58.588 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:58.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 >> v1:172.21.15.155:6816/49624 conn(0x7fbf980475f0 legacy=0x7fbf98049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb412f4f0 msgr2=0x7fbfb41c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb412f4f0 0x7fbfb41c32e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbfac016000 tx=0x7fbfac004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 shutdown_connections 2024-04-25T11:02:58.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbfb412f4f0 0x7fbfb41c32e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfb406eba0 0x7fbfb41c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 --2- 172.21.15.155:0/2256269437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb412ebd0 0x7fbfb41c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 >> 172.21.15.155:0/2256269437 conn(0x7fbfb409dbc0 msgr2=0x7fbfb4129b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:58.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 shutdown_connections 2024-04-25T11:02:58.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.588+0000 7fbfbbd57700 1 -- 172.21.15.155:0/2256269437 wait complete. 2024-04-25T11:02:58.600 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":9,"flags":0,"active_gid":14112,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6816","nonce":49624}]},"active_addr":"172.21.15.155:6816/49624","active_change":"2024-04-25T11:02:39.971472+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":15,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":4160584160}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":2960784954}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":316666483}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.155:0","nonce":2153852605}]}]} 2024-04-25T11:02:58.603 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:02:58.603 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:02:58.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 Processor -- start 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 -- start start 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f3c12b790 0x7f7f3c1308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c1331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c070fc0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3c0703a0 con 0x7f7f3c12b790 2024-04-25T11:02:58.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f435eb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c070c80 con 0x7f7f3c130de0 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f41b88700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c1331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f40b86700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f3c12b790 0x7f7f3c1308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f41387700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7f3c06f4a0 0x7f7f3c12bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35316/0 (socket says 172.21.15.155:35316) 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.722+0000 7f7f41387700 1 -- 172.21.15.155:0/366324012 learned_addr learned my addr 172.21.15.155:0/366324012 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f33fff700 1 -- 172.21.15.155:0/366324012 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774835953 0 0) 0x7f7f3c070fc0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f33fff700 1 -- 172.21.15.155:0/366324012 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f280035b0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f40b86700 1 -- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 msgr2=0x7f7f3c1331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f40b86700 1 --2- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c1331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f40b86700 1 -- 172.21.15.155:0/366324012 >> v1:172.21.15.155:6789/0 conn(0x7f7f3c06f4a0 legacy=0x7f7f3c12bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f40b86700 1 -- 172.21.15.155:0/366324012 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c06f880 con 0x7f7f3c12b790 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f41b88700 1 --2- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c1331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f40b86700 1 --2- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c1308a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009840 tx=0x7f7f2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2694abe43af006c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:58.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f33fff700 1 -- 172.21.15.155:0/366324012 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c011830 con 0x7f7f3c12b790 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f33fff700 1 -- 172.21.15.155:0/366324012 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c011990 con 0x7f7f3c12b790 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f33fff700 1 -- 172.21.15.155:0/366324012 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c011d20 con 0x7f7f3c12b790 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f435eb700 1 -- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 msgr2=0x7f7f3c1308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.723+0000 7f7f435eb700 1 --2- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c1308a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009840 tx=0x7f7f2c009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 -- 172.21.15.155:0/366324012 shutdown_connections 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 --2- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c1331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 --2- 172.21.15.155:0/366324012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c1308a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 -- 172.21.15.155:0/366324012 >> 172.21.15.155:0/366324012 conn(0x7f7f3c09dbc0 msgr2=0x7f7f3c09dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 -- 172.21.15.155:0/366324012 shutdown_connections 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 -- 172.21.15.155:0/366324012 wait complete. 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 Processor -- start 2024-04-25T11:02:58.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.724+0000 7f7f435eb700 1 -- start start 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 0x7f7f3c12e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c12ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c1305b0 con 0x7f7f3c12b790 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c12dba0 con 0x7f7f3c130de0 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c12d180 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 0x7f7f3c12e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41b88700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 0x7f7f3c12e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44414/0 (socket says 172.21.15.155:44414) 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 -- 172.21.15.155:0/3299743298 learned_addr learned my addr 172.21.15.155:0/3299743298 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:58.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41b88700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c12f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40650/0 (socket says 172.21.15.155:40650) 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f40b86700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c12ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 -- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 msgr2=0x7f7f3c12f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c12f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 -- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 msgr2=0x7f7f3c12ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c12ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 -- 172.21.15.155:0/3299743298 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34009a70 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f40b86700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c12ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41b88700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f41387700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 0x7f7f3c12e910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c730 tx=0x7f7f3800ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38005d70 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f380070d0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.725+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2c0094a0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.726+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38007460 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.726+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c1241a0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.726+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f7f380075c0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.727+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f7f3805bcd0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.727+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c12d6a0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.732 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.731+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f7f38017080 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.893+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f3c06f0d0 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.893+0000 7f7f31ffb700 1 -- 172.21.15.155:0/3299743298 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+8812 (secure 0 0 0) 0x7f7f38039980 con 0x7f7f3c06f4a0 2024-04-25T11:02:58.894 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:58.895 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":29,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:02:58.221121+0000","last_up_change":"2024-04-25T11:02:53.034844+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":16,"up_from":22,"up_thru":22,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":51324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":51324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":51324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":51324}]},"public_addr":"172.21.15.155:6800/51324","cluster_addr":"172.21.15.155:6801/51324","heartbeat_back_addr":"172.21.15.155:6803/51324","heartbeat_front_addr":"172.21.15.155:6802/51324","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":23,"up_thru":23,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":51418}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":51418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":51418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":51418}]},"public_addr":"172.21.15.155:6808/51418","cluster_addr":"172.21.15.155:6809/51418","heartbeat_back_addr":"172.21.15.155:6811/51418","heartbeat_front_addr":"172.21.15.155:6810/51418","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":24,"up_thru":25,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":51529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":51529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":51529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":51529}]},"public_addr":"172.21.15.155:6804/51529","cluster_addr":"172.21.15.155:6805/51529","heartbeat_back_addr":"172.21.15.155:6807/51529","heartbeat_front_addr":"172.21.15.155:6806/51529","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":51677}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":51677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":51677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":51677}]},"public_addr":"172.21.15.155:6812/51677","cluster_addr":"172.21.15.155:6813/51677","heartbeat_back_addr":"172.21.15.155:6815/51677","heartbeat_front_addr":"172.21.15.155:6814/51677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":17},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":18},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":19},{"osd":3,"down_stamp":"2024-04-25T11:02:47.002020+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:0/3077341019":"2024-04-26T11:02:39.971273+0000","172.21.15.155:0/966131479":"2024-04-26T11:02:39.971273+0000","172.21.15.155:6816/43298":"2024-04-26T11:02:39.971273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:58.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 >> v1:172.21.15.155:6816/49624 conn(0x7f7f200475f0 legacy=0x7f7f20049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 msgr2=0x7f7f3c12e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 0x7f7f3c12e910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c730 tx=0x7f7f3800ca40 comp rx=0 tx=0).stop 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 shutdown_connections 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c130de0 0x7f7f3c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f7f3c06f4a0 0x7f7f3c12e910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 --2- 172.21.15.155:0/3299743298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c12b790 0x7f7f3c12ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 >> 172.21.15.155:0/3299743298 conn(0x7f7f3c09dbc0 msgr2=0x7f7f3c127eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 shutdown_connections 2024-04-25T11:02:58.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:58.895+0000 7f7f435eb700 1 -- 172.21.15.155:0/3299743298 wait complete. 2024-04-25T11:02:58.908 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:02:58.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.026+0000 7fea42bec700 1 Processor -- start 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.026+0000 7fea42bec700 1 -- start start 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea42bec700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea3c06a900 0x7fea3c094a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea42bec700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c094f80 0x7fea3c095400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea42bec700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c096a90 con 0x7fea3c133210 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea42bec700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c09cec0 con 0x7fea3c06a900 2024-04-25T11:02:59.029 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea42bec700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c09dfa0 con 0x7fea3c094f80 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea40988700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea3c06a900 0x7fea3c094a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea41189700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fea3c133210 0x7fea3c135600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35330/0 (socket says 172.21.15.155:35330) 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c094f80 0x7fea3c095400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea41189700 1 -- 172.21.15.155:0/4093443220 learned_addr learned my addr 172.21.15.155:0/4093443220 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3bfff700 1 -- 172.21.15.155:0/4093443220 >> v2:172.21.15.155:3300/0 conn(0x7fea3c06a900 msgr2=0x7fea3c094a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3bfff700 1 --2- 172.21.15.155:0/4093443220 >> v2:172.21.15.155:3300/0 conn(0x7fea3c06a900 0x7fea3c094a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3bfff700 1 -- 172.21.15.155:0/4093443220 >> v1:172.21.15.155:6789/0 conn(0x7fea3c133210 legacy=0x7fea3c135600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3bfff700 1 -- 172.21.15.155:0/4093443220 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c096cc0 con 0x7fea3c094f80 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3b7fe700 1 -- 172.21.15.155:0/4093443220 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160701892 0 0) 0x7fea3c096a90 con 0x7fea3c133210 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea40988700 1 --2- 172.21.15.155:0/4093443220 >> v2:172.21.15.155:3300/0 conn(0x7fea3c06a900 0x7fea3c094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.027+0000 7fea3bfff700 1 --2- 172.21.15.155:0/4093443220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c094f80 0x7fea3c095400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea28009840 tx=0x7fea28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a8d2bf64a74194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.030 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea3b7fe700 1 -- 172.21.15.155:0/4093443220 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011830 con 0x7fea3c094f80 2024-04-25T11:02:59.031 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea3b7fe700 1 -- 172.21.15.155:0/4093443220 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28011990 con 0x7fea3c094f80 2024-04-25T11:02:59.031 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea3b7fe700 1 -- 172.21.15.155:0/4093443220 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011d20 con 0x7fea3c094f80 2024-04-25T11:02:59.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 -- 172.21.15.155:0/4093443220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c094f80 msgr2=0x7fea3c095400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 --2- 172.21.15.155:0/4093443220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c094f80 0x7fea3c095400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea28009840 tx=0x7fea28009b50 comp rx=0 tx=0).stop 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 -- 172.21.15.155:0/4093443220 shutdown_connections 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 --2- 172.21.15.155:0/4093443220 >> v2:172.21.15.155:3300/0 conn(0x7fea3c06a900 0x7fea3c094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 --2- 172.21.15.155:0/4093443220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c094f80 0x7fea3c095400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 -- 172.21.15.155:0/4093443220 >> 172.21.15.155:0/4093443220 conn(0x7fea3c09d480 msgr2=0x7fea3c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 -- 172.21.15.155:0/4093443220 shutdown_connections 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.028+0000 7fea42bec700 1 -- 172.21.15.155:0/4093443220 wait complete. 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.029+0000 7fea42bec700 1 Processor -- start 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.029+0000 7fea42bec700 1 -- start start 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.029+0000 7fea42bec700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fea3c06a900 0x7fea3c1c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 0x7fea3c1c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c133210 0x7fea3c1c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c070040 con 0x7fea3c094f80 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c136810 con 0x7fea3c133210 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c070350 con 0x7fea3c06a900 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea40988700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fea3c06a900 0x7fea3c1c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 0x7fea3c1c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea40988700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fea3c06a900 0x7fea3c1c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44416/0 (socket says 172.21.15.155:44416) 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea40988700 1 -- 172.21.15.155:0/3867820883 learned_addr learned my addr 172.21.15.155:0/3867820883 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 0x7fea3c1c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40608/0 (socket says 172.21.15.155:40608) 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea41189700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c133210 0x7fea3c1c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 -- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fea3c06a900 msgr2=0x7fea3c1c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fea3c06a900 0x7fea3c1c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 -- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c133210 msgr2=0x7fea3c1c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c133210 0x7fea3c1c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 -- 172.21.15.155:0/3867820883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea300094a0 con 0x7fea3c094f80 2024-04-25T11:02:59.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea41189700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c133210 0x7fea3c1c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea3bfff700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 0x7fea3c1c36d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea3c06d8d0 tx=0x7fea28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28004340 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2801f070 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea280094a0 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea280036a0 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.030+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c092fa0 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.031+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c062510 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.031+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fea28021030 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.031+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fea2805d8a0 con 0x7fea3c094f80 2024-04-25T11:02:59.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.035+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fea280253a0 con 0x7fea3c094f80 2024-04-25T11:02:59.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.198+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea3c06f700 con 0x7fea3c094f80 2024-04-25T11:02:59.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.199+0000 7fea397fa700 1 -- 172.21.15.155:0/3867820883 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+8812 (secure 0 0 0) 0x7fea3c06f700 con 0x7fea3c094f80 2024-04-25T11:02:59.200 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:02:59.200 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":29,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:02:58.221121+0000","last_up_change":"2024-04-25T11:02:53.034844+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":16,"up_from":22,"up_thru":22,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":51324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":51324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":51324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":51324}]},"public_addr":"172.21.15.155:6800/51324","cluster_addr":"172.21.15.155:6801/51324","heartbeat_back_addr":"172.21.15.155:6803/51324","heartbeat_front_addr":"172.21.15.155:6802/51324","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":23,"up_thru":23,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":51418}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":51418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":51418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":51418}]},"public_addr":"172.21.15.155:6808/51418","cluster_addr":"172.21.15.155:6809/51418","heartbeat_back_addr":"172.21.15.155:6811/51418","heartbeat_front_addr":"172.21.15.155:6810/51418","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":24,"up_thru":25,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":51529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":51529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":51529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":51529}]},"public_addr":"172.21.15.155:6804/51529","cluster_addr":"172.21.15.155:6805/51529","heartbeat_back_addr":"172.21.15.155:6807/51529","heartbeat_front_addr":"172.21.15.155:6806/51529","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":51677}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":51677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":51677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":51677}]},"public_addr":"172.21.15.155:6812/51677","cluster_addr":"172.21.15.155:6813/51677","heartbeat_back_addr":"172.21.15.155:6815/51677","heartbeat_front_addr":"172.21.15.155:6814/51677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":17},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":18},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":19},{"osd":3,"down_stamp":"2024-04-25T11:02:47.002020+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:0/3077341019":"2024-04-26T11:02:39.971273+0000","172.21.15.155:0/966131479":"2024-04-26T11:02:39.971273+0000","172.21.15.155:6816/43298":"2024-04-26T11:02:39.971273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:02:59.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 >> v1:172.21.15.155:6816/49624 conn(0x7fea200475f0 legacy=0x7fea20049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 msgr2=0x7fea3c1c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 0x7fea3c1c36d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea3c06d8d0 tx=0x7fea28004750 comp rx=0 tx=0).stop 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 shutdown_connections 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c133210 0x7fea3c1c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fea3c06a900 0x7fea3c1c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 --2- 172.21.15.155:0/3867820883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c094f80 0x7fea3c1c36d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 >> 172.21.15.155:0/3867820883 conn(0x7fea3c09d480 msgr2=0x7fea3c12f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 shutdown_connections 2024-04-25T11:02:59.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.201+0000 7fea42bec700 1 -- 172.21.15.155:0/3867820883 wait complete. 2024-04-25T11:02:59.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:02:59.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:02:59.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:02:59.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:02:59.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 Processor -- start 2024-04-25T11:02:59.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 -- start start 2024-04-25T11:02:59.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 Processor -- start 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 -- start start 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006e400 0x7fbe5006e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8094a90 0x7fc3c8094e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3c8096140 0x7fc3c80953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8093080 0x7fc3c80934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c809e010 con 0x7fc3c8096140 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c8096a10 con 0x7fc3c8094a90 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fc3cd1c6700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80958f0 con 0x7fc3c8093080 2024-04-25T11:02:59.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c659c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3c8096140 0x7fc3c80953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c659c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3c8096140 0x7fc3c80953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40614/0 (socket says 172.21.15.155:40614) 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c659c700 1 -- 172.21.15.155:0/3582463134 learned_addr learned my addr 172.21.15.155:0/3582463134 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c759e700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8093080 0x7fc3c80934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006f140 0x7fbe5012f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe50072780 con 0x7fbe5006ed20 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5006ca20 con 0x7fbe5006e400 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.344+0000 7fbe552c5700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5006fb80 con 0x7fbe5006f140 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4f7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006f140 0x7fbe5012f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4e7fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fbe5006ed20 0x7fbe5012ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35346/0 (socket says 172.21.15.155:35346) 2024-04-25T11:02:59.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4e7fc700 1 -- 172.21.15.155:0/1916866527 learned_addr learned my addr 172.21.15.155:0/1916866527 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:59.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c6d9d700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8094a90 0x7fc3c8094e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006e400 msgr2=0x7fbe5006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4f7fe700 1 --2- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006e400 0x7fbe5006e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c759e700 1 -- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8094a90 msgr2=0x7fc3c8094e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c759e700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8094a90 0x7fc3c8094e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c759e700 1 -- 172.21.15.155:0/3582463134 >> v2:172.21.15.155:3300/0 conn(0x7fc3c8096140 msgr2=0x7fc3c80953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c759e700 1 --2- 172.21.15.155:0/3582463134 >> v2:172.21.15.155:3300/0 conn(0x7fc3c8096140 0x7fc3c80953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fc3c759e700 1 -- 172.21.15.155:0/3582463134 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c806ab70 con 0x7fc3c8093080 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/1916866527 >> v1:172.21.15.155:6789/0 conn(0x7fbe5006ed20 legacy=0x7fbe5012ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/1916866527 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5006c6e0 con 0x7fbe5006f140 2024-04-25T11:02:59.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.345+0000 7fbe4f7fe700 1 --2- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006f140 0x7fbe5012f170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe48014a60 tx=0x7fbe48014d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76861748e0e2635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fc3c759e700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8093080 0x7fc3c80934d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009d90 tx=0x7fc3c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917b8b86c61ecacc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe4dffb700 1 -- 172.21.15.155:0/1916866527 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4800fd50 con 0x7fbe5006f140 2024-04-25T11:02:59.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fc3c5d9b700 1 -- 172.21.15.155:0/3582463134 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000bd20 con 0x7fc3c8093080 2024-04-25T11:02:59.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe4dffb700 1 -- 172.21.15.155:0/1916866527 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe480138a0 con 0x7fbe5006f140 2024-04-25T11:02:59.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe4dffb700 1 -- 172.21.15.155:0/1916866527 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48013c50 con 0x7fbe5006f140 2024-04-25T11:02:59.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fc3c5d9b700 1 -- 172.21.15.155:0/3582463134 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c000be80 con 0x7fc3c8093080 2024-04-25T11:02:59.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fc3c5d9b700 1 -- 172.21.15.155:0/3582463134 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0010710 con 0x7fc3c8093080 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe552c5700 1 -- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006f140 msgr2=0x7fbe5012f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe552c5700 1 --2- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006f140 0x7fbe5012f170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe48014a60 tx=0x7fbe48014d70 comp rx=0 tx=0).stop 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe552c5700 1 -- 172.21.15.155:0/1916866527 shutdown_connections 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe552c5700 1 --2- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006f140 0x7fbe5012f170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe552c5700 1 --2- 172.21.15.155:0/1916866527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006e400 0x7fbe5006e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.346+0000 7fbe552c5700 1 -- 172.21.15.155:0/1916866527 >> 172.21.15.155:0/1916866527 conn(0x7fbe5009dac0 msgr2=0x7fbe5009ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fbe552c5700 1 -- 172.21.15.155:0/1916866527 shutdown_connections 2024-04-25T11:02:59.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fbe552c5700 1 -- 172.21.15.155:0/1916866527 wait complete. 2024-04-25T11:02:59.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8093080 msgr2=0x7fc3c80934d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8093080 0x7fc3c80934d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009d90 tx=0x7fc3c000b330 comp rx=0 tx=0).stop 2024-04-25T11:02:59.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/3582463134 shutdown_connections 2024-04-25T11:02:59.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8093080 0x7fc3c80934d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 --2- 172.21.15.155:0/3582463134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8094a90 0x7fc3c8094e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 --2- 172.21.15.155:0/3582463134 >> v2:172.21.15.155:3300/0 conn(0x7fc3c8096140 0x7fc3c80953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/3582463134 >> 172.21.15.155:0/3582463134 conn(0x7fc3c809d480 msgr2=0x7fc3c809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/3582463134 shutdown_connections 2024-04-25T11:02:59.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.347+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/3582463134 wait complete. 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 Processor -- start 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 -- start start 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe5006e400 0x7fbe50094950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006ed20 0x7fbe50094e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006f140 0x7fbe501bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500962f0 con 0x7fbe5006e400 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50122f90 con 0x7fbe5006f140 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe552c5700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50122ce0 con 0x7fbe5006ed20 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe4f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006f140 0x7fbe501bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe4f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006f140 0x7fbe501bedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40698/0 (socket says 172.21.15.155:40698) 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/184985793 learned_addr learned my addr 172.21.15.155:0/184985793 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe4e7fc700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006ed20 0x7fbe50094e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fc3cd1c6700 1 Processor -- start 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.348+0000 7fbe4effd700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe5006e400 0x7fbe50094950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006ed20 msgr2=0x7fbe50094e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4f7fe700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006ed20 0x7fbe50094e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe5006e400 msgr2=0x7fbe50094950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4f7fe700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe5006e400 0x7fbe50094950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/184985793 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48014660 con 0x7fbe5006f140 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4e7fc700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fbe5006ed20 0x7fbe50094e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe4f7fe700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006f140 0x7fbe501bedd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe4801b000 tx=0x7fbe48004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48014240 con 0x7fbe5006f140 2024-04-25T11:02:59.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48003e50 con 0x7fbe5006f140 2024-04-25T11:02:59.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48004270 con 0x7fbe5006f140 2024-04-25T11:02:59.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 -- start start 2024-04-25T11:02:59.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c8093080 0x7fc3c81bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8094a90 0x7fc3c81bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8096140 0x7fc3c81bef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c8072560 con 0x7fc3c8093080 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c81296f0 con 0x7fc3c8096140 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.349+0000 7fc3cd1c6700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c8123a40 con 0x7fc3c8094a90 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c8093080 0x7fc3c81bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c8093080 0x7fc3c81bbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40622/0 (socket says 172.21.15.155:40622) 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c6d9d700 1 -- 172.21.15.155:0/4082479902 learned_addr learned my addr 172.21.15.155:0/4082479902 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 --2- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8096140 0x7fc3c81bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe50096bb0 con 0x7fbe5006f140 2024-04-25T11:02:59.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe500953d0 con 0x7fbe5006f140 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c659c700 1 --2- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8094a90 0x7fc3c81bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 -- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8094a90 msgr2=0x7fc3c81bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 --2- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc3c8094a90 0x7fc3c81bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 -- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c8093080 msgr2=0x7fc3c81bbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 --2- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c8093080 0x7fc3c81bbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 -- 172.21.15.155:0/4082479902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c0009a70 con 0x7fc3c8096140 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3c759e700 1 --2- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8096140 0x7fc3c81bef00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009d90 tx=0x7fc3c0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0010d20 con 0x7fc3c8096140 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c0003e10 con 0x7fc3c8096140 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0010ea0 con 0x7fc3c8096140 2024-04-25T11:02:59.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c81233a0 con 0x7fc3c8096140 2024-04-25T11:02:59.362 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.350+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c81228a0 con 0x7fc3c8096140 2024-04-25T11:02:59.362 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.351+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe5008b000 con 0x7fbe5006f140 2024-04-25T11:02:59.362 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.351+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fbe48023070 con 0x7fbe5006f140 2024-04-25T11:02:59.362 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.351+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3c8127710 con 0x7fc3c8096140 2024-04-25T11:02:59.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.351+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fbe480665d0 con 0x7fbe5006f140 2024-04-25T11:02:59.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.352+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 --> v1:172.21.15.155:6804/51529 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe30051a40 con 0x7fbe3004dd70 2024-04-25T11:02:59.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.352+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe48068c90 con 0x7fbe5006f140 2024-04-25T11:02:59.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.352+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fc3c0004280 con 0x7fc3c8096140 2024-04-25T11:02:59.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.352+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fc3c0062780 con 0x7fc3c8096140 2024-04-25T11:02:59.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.352+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 --> v1:172.21.15.155:6808/51418 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3b4051a40 con 0x7fc3b404dd70 2024-04-25T11:02:59.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.352+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c0062bd0 con 0x7fc3c8096140 2024-04-25T11:02:59.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.354+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== osd.1 v1:172.21.15.155:6808/51418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7fc3b4051a40 con 0x7fc3b404dd70 2024-04-25T11:02:59.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.357+0000 7f3e8d1d9700 1 Processor -- start 2024-04-25T11:02:59.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.357+0000 7f472b69b700 1 Processor -- start 2024-04-25T11:02:59.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.357+0000 7f3e8d1d9700 1 -- start start 2024-04-25T11:02:59.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== osd.2 v1:172.21.15.155:6804/51529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7fbe30051a40 con 0x7fbe3004dd70 2024-04-25T11:02:59.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8d1d9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e88131dc0 0x7f3e881321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8d1d9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e8806cf80 0x7f3e8813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8d1d9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8808c500 con 0x7f3e8806c470 2024-04-25T11:02:59.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8d1d9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8808bf40 con 0x7f3e88131dc0 2024-04-25T11:02:59.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8d1d9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88097e90 con 0x7f3e8806cf80 2024-04-25T11:02:59.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e86d9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e88131dc0 0x7f3e881321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8659c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3e8806c470 0x7f3e8806c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35358/0 (socket says 172.21.15.155:35358) 2024-04-25T11:02:59.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e8659c700 1 -- 172.21.15.155:0/2433687122 learned_addr learned my addr 172.21.15.155:0/2433687122 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:59.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 -- start start 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4724094d20 0x7f4724095100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4724095640 0x7f4724095a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472412f1d0 0x7f47241315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472409ce50 con 0x7f4724095640 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240974f0 con 0x7f472412f1d0 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f472b69b700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4724095fa0 con 0x7f4724094d20 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f4728c36700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4724095640 0x7f4724095a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f4728c36700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4724095640 0x7f4724095a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40636/0 (socket says 172.21.15.155:40636) 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f4728c36700 1 -- 172.21.15.155:0/1738025267 learned_addr learned my addr 172.21.15.155:0/1738025267 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 --2- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472412f1d0 0x7f47241315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 -- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4724094d20 msgr2=0x7f4724095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 --2- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4724094d20 0x7f4724095100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 -- 172.21.15.155:0/1738025267 >> v2:172.21.15.155:3300/0 conn(0x7f4724095640 msgr2=0x7f4724095a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 --2- 172.21.15.155:0/1738025267 >> v2:172.21.15.155:3300/0 conn(0x7f4724095640 0x7f4724095a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 -- 172.21.15.155:0/1738025267 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724096940 con 0x7f472412f1d0 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f4729c38700 1 --2- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472412f1d0 0x7f47241315c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f471c00bfa0 tx=0x7f471c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79a6a2566f3ef588 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e85d9b700 1 -- 172.21.15.155:0/2433687122 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345898841 0 0) 0x7f3e8808c500 con 0x7f3e8806c470 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.358+0000 7f3e85d9b700 1 -- 172.21.15.155:0/2433687122 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e700035b0 con 0x7f3e8806c470 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e86d9d700 1 -- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e8806cf80 msgr2=0x7f3e8813ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e86d9d700 1 --2- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e8806cf80 0x7f3e8813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e86d9d700 1 -- 172.21.15.155:0/2433687122 >> v1:172.21.15.155:6789/0 conn(0x7f3e8806c470 legacy=0x7f3e8806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e86d9d700 1 -- 172.21.15.155:0/2433687122 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880727c0 con 0x7f3e88131dc0 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e86d9d700 1 --2- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e88131dc0 0x7f3e881321a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009870 tx=0x7f3e78009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e441d4fbab0da3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e85d9b700 1 -- 172.21.15.155:0/2433687122 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78011830 con 0x7f3e88131dc0 2024-04-25T11:02:59.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e85d9b700 1 -- 172.21.15.155:0/2433687122 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e78011990 con 0x7f3e88131dc0 2024-04-25T11:02:59.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.359+0000 7f3e85d9b700 1 -- 172.21.15.155:0/2433687122 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78011d00 con 0x7f3e88131dc0 2024-04-25T11:02:59.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f471bfff700 1 -- 172.21.15.155:0/1738025267 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00de00 con 0x7f472412f1d0 2024-04-25T11:02:59.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f471bfff700 1 -- 172.21.15.155:0/1738025267 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c00b020 con 0x7f472412f1d0 2024-04-25T11:02:59.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f471bfff700 1 -- 172.21.15.155:0/1738025267 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00b3d0 con 0x7f472412f1d0 2024-04-25T11:02:59.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e88131dc0 msgr2=0x7f3e881321a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f3e8d1d9700 1 --2- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e88131dc0 0x7f3e881321a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009870 tx=0x7f3e78009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:59.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/2433687122 shutdown_connections 2024-04-25T11:02:59.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f3e8d1d9700 1 --2- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e8806cf80 0x7f3e8813ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f3e8d1d9700 1 --2- 172.21.15.155:0/2433687122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e88131dc0 0x7f3e881321a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/2433687122 >> 172.21.15.155:0/2433687122 conn(0x7f3e8809dac0 msgr2=0x7f3e8809ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 -- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472412f1d0 msgr2=0x7f47241315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 --2- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472412f1d0 0x7f47241315c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f471c00bfa0 tx=0x7f471c00d330 comp rx=0 tx=0).stop 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 -- 172.21.15.155:0/1738025267 shutdown_connections 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 --2- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472412f1d0 0x7f47241315c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 --2- 172.21.15.155:0/1738025267 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4724094d20 0x7f4724095100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 --2- 172.21.15.155:0/1738025267 >> v2:172.21.15.155:3300/0 conn(0x7f4724095640 0x7f4724095a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.360+0000 7f472b69b700 1 -- 172.21.15.155:0/1738025267 >> 172.21.15.155:0/1738025267 conn(0x7f472409d480 msgr2=0x7f472409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.361+0000 7f472b69b700 1 -- 172.21.15.155:0/1738025267 shutdown_connections 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.361+0000 7f472b69b700 1 -- 172.21.15.155:0/1738025267 wait complete. 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.361+0000 7f472b69b700 1 Processor -- start 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.361+0000 7f472b69b700 1 -- start start 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f472b69b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724094d20 0x7f4724091320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f472b69b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724095640 0x7f4724091860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f472b69b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f472412f1d0 0x7f4724091da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f472b69b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472408fb80 con 0x7f4724094d20 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f472b69b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4724090df0 con 0x7f4724095640 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f472b69b700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f472408fe70 con 0x7f472412f1d0 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f4729437700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724094d20 0x7f4724091320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f4729437700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724094d20 0x7f4724091320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40646/0 (socket says 172.21.15.155:40646) 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f4729437700 1 -- 172.21.15.155:0/3703261315 learned_addr learned my addr 172.21.15.155:0/3703261315 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f4729c38700 1 --2- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f472412f1d0 0x7f4724091da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.362+0000 7f4728c36700 1 --2- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724095640 0x7f4724091860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/2433687122 shutdown_connections 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4729437700 1 -- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f472412f1d0 msgr2=0x7f4724091da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4729437700 1 --2- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f472412f1d0 0x7f4724091da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4729437700 1 -- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724095640 msgr2=0x7f4724091860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4729437700 1 --2- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724095640 0x7f4724091860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4729437700 1 -- 172.21.15.155:0/3703261315 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c00bc80 con 0x7f4724094d20 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4729437700 1 --2- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724094d20 0x7f4724091320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f472000b530 tx=0x7f472000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720011830 con 0x7f4724094d20 2024-04-25T11:02:59.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720011990 con 0x7f4724094d20 2024-04-25T11:02:59.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720011d40 con 0x7f4724094d20 2024-04-25T11:02:59.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.363+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/2433687122 wait complete. 2024-04-25T11:02:59.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.364+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472412d3d0 con 0x7f4724094d20 2024-04-25T11:02:59.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.364+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472408d710 con 0x7f4724094d20 2024-04-25T11:02:59.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.364+0000 7f3e8d1d9700 1 Processor -- start 2024-04-25T11:02:59.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.364+0000 7f3e8d1d9700 1 -- start start 2024-04-25T11:02:59.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4724062510 con 0x7f4724094d20 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8d1d9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8806c470 0x7f3e88132a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8d1d9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e8806cf80 0x7f3e88132fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8d1d9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e88131dc0 0x7f3e881334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8d1d9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88134cc0 con 0x7f3e8806c470 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8d1d9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8813ae80 con 0x7f3e8806cf80 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8d1d9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881350b0 con 0x7f3e88131dc0 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8759e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e88131dc0 0x7f3e881334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8759e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e88131dc0 0x7f3e881334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44472/0 (socket says 172.21.15.155:44472) 2024-04-25T11:02:59.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e8759e700 1 -- 172.21.15.155:0/1399033817 learned_addr learned my addr 172.21.15.155:0/1399033817 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.365+0000 7f3e86d9d700 1 --2- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8806c470 0x7f3e88132a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e8759e700 1 -- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e8806cf80 msgr2=0x7f3e88132fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e8759e700 1 --2- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e8806cf80 0x7f3e88132fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e8759e700 1 -- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8806c470 msgr2=0x7f3e88132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e8759e700 1 --2- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8806c470 0x7f3e88132a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e8759e700 1 -- 172.21.15.155:0/1399033817 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780094a0 con 0x7f3e88131dc0 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e86d9d700 1 --2- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8806c470 0x7f3e88132a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e8759e700 1 --2- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e88131dc0 0x7f3e881334f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000b970 tx=0x7f3e8000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8000c9b0 con 0x7f3e88131dc0 2024-04-25T11:02:59.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8000cb30 con 0x7f3e88131dc0 2024-04-25T11:02:59.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f472001a400 con 0x7f4724094d20 2024-04-25T11:02:59.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f472005cde0 con 0x7f4724094d20 2024-04-25T11:02:59.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 --> v1:172.21.15.155:6800/51324 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4708051b90 con 0x7f470804dec0 2024-04-25T11:02:59.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f472005d190 con 0x7f4724094d20 2024-04-25T11:02:59.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.366+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80014440 con 0x7f3e88131dc0 2024-04-25T11:02:59.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.367+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e881343b0 con 0x7f3e88131dc0 2024-04-25T11:02:59.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.367+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e881c7650 con 0x7f3e88131dc0 2024-04-25T11:02:59.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.367+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e8813b390 con 0x7f3e88131dc0 2024-04-25T11:02:59.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.368+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== osd.0 v1:172.21.15.155:6800/51324 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7f4708051b90 con 0x7f470804dec0 2024-04-25T11:02:59.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.368+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3e8001d3b0 con 0x7f3e88131dc0 2024-04-25T11:02:59.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.369+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f3e80006e40 con 0x7f3e88131dc0 2024-04-25T11:02:59.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.369+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 --> v1:172.21.15.155:6812/51677 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e74051a40 con 0x7f3e7404dd70 2024-04-25T11:02:59.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.369+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e8005d010 con 0x7f3e88131dc0 2024-04-25T11:02:59.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.372+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== osd.3 v1:172.21.15.155:6812/51677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7f3e74051a40 con 0x7f3e7404dd70 2024-04-25T11:02:59.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.373+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 --> v1:172.21.15.155:6808/51418 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3c8067020 con 0x7fc3b404dd70 2024-04-25T11:02:59.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.373+0000 7fc3af7fe700 1 -- 172.21.15.155:0/4082479902 <== osd.1 v1:172.21.15.155:6808/51418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1793987854) 0x7fc3c8067020 con 0x7fc3b404dd70 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.374+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 >> v1:172.21.15.155:6808/51418 conn(0x7fc3b404dd70 legacy=0x7fc3b40501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.374+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 >> v1:172.21.15.155:6816/49624 conn(0x7fc3b40475f0 legacy=0x7fc3b4049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.374+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8096140 msgr2=0x7fc3c81bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.374+0000 7fc3cd1c6700 1 --2- 172.21.15.155:0/4082479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3c8096140 0x7fc3c81bef00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009d90 tx=0x7fc3c0004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.374+0000 7fc3c759e700 1 -- 172.21.15.155:0/4082479902 reap_dead start 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.377+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 shutdown_connections 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.377+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 >> 172.21.15.155:0/4082479902 conn(0x7fc3c809d480 msgr2=0x7fc3c8097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.378+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 shutdown_connections 2024-04-25T11:02:59.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.378+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 --> v1:172.21.15.155:6804/51529 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe50097130 con 0x7fbe3004dd70 2024-04-25T11:02:59.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.378+0000 7fc3cd1c6700 1 -- 172.21.15.155:0/4082479902 wait complete. 2024-04-25T11:02:59.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.379+0000 7fbe477fe700 1 -- 172.21.15.155:0/184985793 <== osd.2 v1:172.21.15.155:6804/51529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1720373281) 0x7fbe50097130 con 0x7fbe3004dd70 2024-04-25T11:02:59.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.379+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 >> v1:172.21.15.155:6804/51529 conn(0x7fbe3004dd70 legacy=0x7fbe300501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.379+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 >> v1:172.21.15.155:6816/49624 conn(0x7fbe300475f0 legacy=0x7fbe30049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.379+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006f140 msgr2=0x7fbe501bedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.379+0000 7fbe552c5700 1 --2- 172.21.15.155:0/184985793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe5006f140 0x7fbe501bedd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe4801b000 tx=0x7fbe48004ad0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.380+0000 7fbe4f7fe700 1 -- 172.21.15.155:0/184985793 reap_dead start 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.380+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 shutdown_connections 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.380+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 >> 172.21.15.155:0/184985793 conn(0x7fbe5009dac0 msgr2=0x7fbe5008d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.380+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 shutdown_connections 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.380+0000 7fbe552c5700 1 -- 172.21.15.155:0/184985793 wait complete. 2024-04-25T11:02:59.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.384+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 --> v1:172.21.15.155:6800/51324 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f472406add0 con 0x7f470804dec0 2024-04-25T11:02:59.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f4719ffb700 1 -- 172.21.15.155:0/3703261315 <== osd.0 v1:172.21.15.155:6800/51324 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2115957317) 0x7f472406add0 con 0x7f470804dec0 2024-04-25T11:02:59.389 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 >> v1:172.21.15.155:6800/51324 conn(0x7f470804dec0 legacy=0x7f4708050320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 >> v1:172.21.15.155:6816/49624 conn(0x7f47080476c0 legacy=0x7f4708049b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724094d20 msgr2=0x7f4724091320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f472b69b700 1 --2- 172.21.15.155:0/3703261315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724094d20 0x7f4724091320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f472000b530 tx=0x7f472000b840 comp rx=0 tx=0).stop 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f4729c38700 1 -- 172.21.15.155:0/3703261315 reap_dead start 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 shutdown_connections 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.385+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 >> 172.21.15.155:0/3703261315 conn(0x7f472409d480 msgr2=0x7f472412e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.386+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 shutdown_connections 2024-04-25T11:02:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.386+0000 7f472b69b700 1 -- 172.21.15.155:0/3703261315 wait complete. 2024-04-25T11:02:59.393 INFO:teuthology.orchestra.run.smithi155.stdout:98784247811 2024-04-25T11:02:59.393 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:02:59.395 INFO:teuthology.orchestra.run.smithi155.stdout:103079215107 2024-04-25T11:02:59.395 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:02:59.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.396+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 --> v1:172.21.15.155:6812/51677 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e88135440 con 0x7f3e7404dd70 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.396+0000 7f3e6f7fe700 1 -- 172.21.15.155:0/1399033817 <== osd.3 v1:172.21.15.155:6812/51677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 4081650579) 0x7f3e88135440 con 0x7f3e7404dd70 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 >> v1:172.21.15.155:6812/51677 conn(0x7f3e7404dd70 legacy=0x7f3e740501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 >> v1:172.21.15.155:6816/49624 conn(0x7f3e740475f0 legacy=0x7f3e74049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e88131dc0 msgr2=0x7f3e881334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 --2- 172.21.15.155:0/1399033817 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3e88131dc0 0x7f3e881334f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000b970 tx=0x7f3e8000bd30 comp rx=0 tx=0).stop 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8759e700 1 -- 172.21.15.155:0/1399033817 reap_dead start 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 shutdown_connections 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 >> 172.21.15.155:0/1399033817 conn(0x7f3e8809dac0 msgr2=0x7f3e8808d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 shutdown_connections 2024-04-25T11:02:59.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.397+0000 7f3e8d1d9700 1 -- 172.21.15.155:0/1399033817 wait complete. 2024-04-25T11:02:59.404 INFO:teuthology.orchestra.run.smithi155.stdout:94489280516 2024-04-25T11:02:59.404 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:02:59.412 INFO:teuthology.orchestra.run.smithi155.stdout:107374182403 2024-04-25T11:02:59.413 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:02:59.533 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 Processor -- start 2024-04-25T11:02:59.534 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 -- start start 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412e770 0x7f794412ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412b940 0x7f7944136b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794412f640 con 0x7f794412f1c0 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794406fc40 con 0x7f794412b940 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.533+0000 7f79490f6700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f794406f2b0 con 0x7f794412e770 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794359e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412b940 0x7f7944136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794259c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f794412f1c0 0x7f794412b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35372/0 (socket says 172.21.15.155:35372) 2024-04-25T11:02:59.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794259c700 1 -- 172.21.15.155:0/1599974840 learned_addr learned my addr 172.21.15.155:0/1599974840 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f7942d9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412e770 0x7f794412ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794359e700 1 -- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412e770 msgr2=0x7f794412ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794359e700 1 --2- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412e770 0x7f794412ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794359e700 1 -- 172.21.15.155:0/1599974840 >> v1:172.21.15.155:6789/0 conn(0x7f794412f1c0 legacy=0x7f794412b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794359e700 1 -- 172.21.15.155:0/1599974840 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794406f470 con 0x7f794412b940 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f7942d9d700 1 --2- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412e770 0x7f794412ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f794359e700 1 --2- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412b940 0x7f7944136b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f793c009dc0 tx=0x7f793c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118f9dc2913e7563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f7941d9b700 1 -- 172.21.15.155:0/1599974840 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00e070 con 0x7f794412b940 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f7941d9b700 1 -- 172.21.15.155:0/1599974840 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c00bb20 con 0x7f794412b940 2024-04-25T11:02:59.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.534+0000 7f7941d9b700 1 -- 172.21.15.155:0/1599974840 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c012430 con 0x7f794412b940 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.535+0000 7f79490f6700 1 -- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412b940 msgr2=0x7f7944136b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.535+0000 7f79490f6700 1 --2- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412b940 0x7f7944136b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f793c009dc0 tx=0x7f793c00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.535+0000 7f79490f6700 1 -- 172.21.15.155:0/1599974840 shutdown_connections 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.535+0000 7f79490f6700 1 --2- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412e770 0x7f794412ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.535+0000 7f79490f6700 1 --2- 172.21.15.155:0/1599974840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412b940 0x7f7944136b10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.535+0000 7f79490f6700 1 -- 172.21.15.155:0/1599974840 >> 172.21.15.155:0/1599974840 conn(0x7f79440a0c70 msgr2=0x7f79440a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.536+0000 7f79490f6700 1 -- 172.21.15.155:0/1599974840 shutdown_connections 2024-04-25T11:02:59.537 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.536+0000 7f79490f6700 1 -- 172.21.15.155:0/1599974840 wait complete. 2024-04-25T11:02:59.538 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.536+0000 7fa84b9e4700 1 Processor -- start 2024-04-25T11:02:59.538 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.536+0000 7fa84b9e4700 1 -- start start 2024-04-25T11:02:59.538 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 Processor -- start 2024-04-25T11:02:59.539 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 -- start start 2024-04-25T11:02:59.539 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa84b9e4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa84409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.539 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa84b9e4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84406d4d0 0x7fa84409bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.539 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa84b9e4700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa844132b60 0x7fa844133010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa84b9e4700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84406dd70 con 0x7fa844132b60 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa84b9e4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84409c430 con 0x7fa84406d4d0 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa84b9e4700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa84409ad20 con 0x7fa84409b5d0 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794412b940 0x7f794408fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 0x7f7944090290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412f1c0 0x7f79440907d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944093230 con 0x7f794412b940 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7944091eb0 con 0x7f794412e770 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f79490f6700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79440925d0 con 0x7f794412f1c0 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f794259c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 0x7f7944090290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f794259c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 0x7f7944090290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40740/0 (socket says 172.21.15.155:40740) 2024-04-25T11:02:59.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f794259c700 1 -- 172.21.15.155:0/2465454915 learned_addr learned my addr 172.21.15.155:0/2465454915 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7f7942d9d700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794412b940 0x7f794408fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa849f81700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa844132b60 0x7fa844133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa849f81700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa844132b60 0x7fa844133010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40674/0 (socket says 172.21.15.155:40674) 2024-04-25T11:02:59.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.537+0000 7fa849f81700 1 -- 172.21.15.155:0/2181923949 learned_addr learned my addr 172.21.15.155:0/2181923949 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa848f7f700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84406d4d0 0x7fa84409bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f794259c700 1 -- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412f1c0 msgr2=0x7f79440907d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f794259c700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412f1c0 0x7f79440907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f794259c700 1 -- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794412b940 msgr2=0x7f794408fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f794259c700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794412b940 0x7f794408fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f794259c700 1 -- 172.21.15.155:0/2465454915 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340094a0 con 0x7f794412e770 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa849f81700 1 -- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 msgr2=0x7fa84409b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa849f81700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa84409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa849f81700 1 -- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84406d4d0 msgr2=0x7fa84409bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa849f81700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84406d4d0 0x7fa84409bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa849f81700 1 -- 172.21.15.155:0/2181923949 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84406e010 con 0x7fa844132b60 2024-04-25T11:02:59.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa849f81700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa844132b60 0x7fa844133010 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009e80 tx=0x7fa83c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb5e697bf94d21e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f794259c700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 0x7f7944090290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f793800c500 tx=0x7f793800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa83bfff700 1 -- 172.21.15.155:0/2181923949 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00e070 con 0x7fa844132b60 2024-04-25T11:02:59.544 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938005d70 con 0x7f794412e770 2024-04-25T11:02:59.544 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c009a70 con 0x7f794412e770 2024-04-25T11:02:59.544 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa83bfff700 1 -- 172.21.15.155:0/2181923949 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c00b8d0 con 0x7fa844132b60 2024-04-25T11:02:59.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.538+0000 7fa83bfff700 1 -- 172.21.15.155:0/2181923949 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00bc80 con 0x7fa844132b60 2024-04-25T11:02:59.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.539+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794412c0b0 con 0x7f794412e770 2024-04-25T11:02:59.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.539+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380070d0 con 0x7f794412e770 2024-04-25T11:02:59.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.539+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938007440 con 0x7f794412e770 2024-04-25T11:02:59.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.539+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79240052e0 con 0x7f794412e770 2024-04-25T11:02:59.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.540+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f79380075a0 con 0x7f794412e770 2024-04-25T11:02:59.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.541+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f793805bfb0 con 0x7f794412e770 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.539+0000 7fa84b9e4700 1 -- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa844132b60 msgr2=0x7fa844133010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.539+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa844132b60 0x7fa844133010 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009e80 tx=0x7fa83c00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 -- 172.21.15.155:0/2181923949 shutdown_connections 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa844132b60 0x7fa844133010 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa84409b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/2181923949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84406d4d0 0x7fa84409bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 -- 172.21.15.155:0/2181923949 >> 172.21.15.155:0/2181923949 conn(0x7fa84409d8b0 msgr2=0x7fa84409dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 -- 172.21.15.155:0/2181923949 shutdown_connections 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.543+0000 7fa84b9e4700 1 -- 172.21.15.155:0/2181923949 wait complete. 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 Processor -- start 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 -- start start 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84406d4d0 0x7fa8441c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa8441c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa844132b60 0x7fa8441c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844139b20 con 0x7fa84406d4d0 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84411fff0 con 0x7fa844132b60 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa84b9e4700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa84412e830 con 0x7fa84409b5d0 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa849f81700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa844132b60 0x7fa8441c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa849f81700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa844132b60 0x7fa8441c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40756/0 (socket says 172.21.15.155:40756) 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa849f81700 1 -- 172.21.15.155:0/4195923415 learned_addr learned my addr 172.21.15.155:0/4195923415 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.544+0000 7fa848f7f700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa8441c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.549 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f7938039d40 con 0x7f794412e770 2024-04-25T11:02:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa849780700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84406d4d0 0x7fa8441c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa848f7f700 1 -- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa844132b60 msgr2=0x7fa8441c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa848f7f700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa844132b60 0x7fa8441c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa848f7f700 1 -- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84406d4d0 msgr2=0x7fa8441c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa848f7f700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84406d4d0 0x7fa8441c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.550 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa848f7f700 1 -- 172.21.15.155:0/4195923415 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c009b30 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa848f7f700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa8441c45d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa84409d6a0 tx=0x7fa83400ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834013680 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8340137e0 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.545+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84412ead0 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.546+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8441c4b10 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.547+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834013b50 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.547+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa834013cf0 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.547+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fa83405c740 con 0x7fa84409b5d0 2024-04-25T11:02:59.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.548+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8240052e0 con 0x7fa84409b5d0 2024-04-25T11:02:59.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.555+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fa834017020 con 0x7fa84409b5d0 2024-04-25T11:02:59.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 Processor -- start 2024-04-25T11:02:59.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 -- start start 2024-04-25T11:02:59.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00097480 0x7f1c00097860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00134740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c00071f10 con 0x7f1c00128e40 2024-04-25T11:02:59.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c00071d20 con 0x7f1c00129970 2024-04-25T11:02:59.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.582+0000 7f1c052d7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c000721d0 con 0x7f1c00097480 2024-04-25T11:02:59.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfeffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00097480 0x7f1c00097860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.584 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfe7fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1c00128e40 0x7f1c00129260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35388/0 (socket says 172.21.15.155:35388) 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfe7fc700 1 -- 172.21.15.155:0/21086291 learned_addr learned my addr 172.21.15.155:0/21086291 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bff7fe700 1 --2- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfdffb700 1 -- 172.21.15.155:0/21086291 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866595450 0 0) 0x7f1c00071f10 con 0x7f1c00128e40 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfeffd700 1 -- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 msgr2=0x7f1c00134740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfeffd700 1 --2- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00134740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfeffd700 1 -- 172.21.15.155:0/21086291 >> v1:172.21.15.155:6789/0 conn(0x7f1c00128e40 legacy=0x7f1c00129260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfeffd700 1 -- 172.21.15.155:0/21086291 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0006c6e0 con 0x7f1c00097480 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfeffd700 1 --2- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00097480 0x7f1c00097860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009870 tx=0x7f1bf0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=341ea9c541bddc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfdffb700 1 -- 172.21.15.155:0/21086291 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0011830 con 0x7f1c00097480 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfdffb700 1 -- 172.21.15.155:0/21086291 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf0011990 con 0x7f1c00097480 2024-04-25T11:02:59.585 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.583+0000 7f1bfdffb700 1 -- 172.21.15.155:0/21086291 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0011d20 con 0x7f1c00097480 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 -- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00097480 msgr2=0x7f1c00097860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 --2- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00097480 0x7f1c00097860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009870 tx=0x7f1bf0009b80 comp rx=0 tx=0).stop 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 -- 172.21.15.155:0/21086291 shutdown_connections 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 --2- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00097480 0x7f1c00097860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 --2- 172.21.15.155:0/21086291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00134740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 -- 172.21.15.155:0/21086291 >> 172.21.15.155:0/21086291 conn(0x7f1c0009dac0 msgr2=0x7f1c0009ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 -- 172.21.15.155:0/21086291 shutdown_connections 2024-04-25T11:02:59.587 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.584+0000 7f1c052d7700 1 -- 172.21.15.155:0/21086291 wait complete. 2024-04-25T11:02:59.587 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.585+0000 7f3189e51700 1 Processor -- start 2024-04-25T11:02:59.587 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.585+0000 7f1c052d7700 1 Processor -- start 2024-04-25T11:02:59.587 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.585+0000 7f1c052d7700 1 -- start start 2024-04-25T11:02:59.588 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.585+0000 7f3189e51700 1 -- start start 2024-04-25T11:02:59.588 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3189e51700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3184094d20 0x7f3184095100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3189e51700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3184095640 0x7f3184095a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3189e51700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 0x7f31841315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3189e51700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318409ce50 con 0x7f3184095640 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3189e51700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840974f0 con 0x7f318412f1d0 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3189e51700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3184095fa0 con 0x7f3184094d20 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 0x7f31841315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 0x7f31841315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40782/0 (socket says 172.21.15.155:40782) 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 -- 172.21.15.155:0/1160715835 learned_addr learned my addr 172.21.15.155:0/1160715835 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3182ffd700 1 --2- 172.21.15.155:0/1160715835 >> v2:172.21.15.155:3300/0 conn(0x7f3184095640 0x7f3184095a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.589 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f31837fe700 1 --2- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3184094d20 0x7f3184095100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1c052d7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 0x7f1c001253c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1c052d7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00125900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1c052d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0008c2e0 0x7f1c00125e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1c052d7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0008b2a0 con 0x7f1c0008c2e0 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1c052d7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0008bf70 con 0x7f1c00129970 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1c052d7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c001281a0 con 0x7f1c00128e40 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1bfeffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 0x7f1c001253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1bfeffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 0x7f1c001253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44538/0 (socket says 172.21.15.155:44538) 2024-04-25T11:02:59.590 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1bfeffd700 1 -- 172.21.15.155:0/367909684 learned_addr learned my addr 172.21.15.155:0/367909684 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 -- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3184094d20 msgr2=0x7f3184095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 --2- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3184094d20 0x7f3184095100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 -- 172.21.15.155:0/1160715835 >> v2:172.21.15.155:3300/0 conn(0x7f3184095640 msgr2=0x7f3184095a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:02:59.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 --2- 172.21.15.155:0/1160715835 >> v2:172.21.15.155:3300/0 conn(0x7f3184095640 0x7f3184095a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3183fff700 1 -- 172.21.15.155:0/1160715835 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184096940 con 0x7f318412f1d0 2024-04-25T11:02:59.591 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f3182ffd700 1 --2- 172.21.15.155:0/1160715835 >> v2:172.21.15.155:3300/0 conn(0x7f3184095640 0x7f3184095a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.586+0000 7f1bff7fe700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0008c2e0 0x7f1c00125e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1bfeffd700 1 -- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 msgr2=0x7f1c00125900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1bfeffd700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00125900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1bfeffd700 1 -- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0008c2e0 msgr2=0x7f1c00125e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1bfeffd700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0008c2e0 0x7f1c00125e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1bfeffd700 1 -- 172.21.15.155:0/367909684 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf00094a0 con 0x7f1c00128e40 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f3183fff700 1 --2- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 0x7f31841315c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f317c009e50 tx=0x7f317c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb15879958dd7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.592 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1bfeffd700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 0x7f1c001253c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009840 tx=0x7f1bf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.593 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f31827fc700 1 -- 172.21.15.155:0/1160715835 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00e070 con 0x7f318412f1d0 2024-04-25T11:02:59.593 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f31827fc700 1 -- 172.21.15.155:0/1160715835 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c00b910 con 0x7f318412f1d0 2024-04-25T11:02:59.593 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf001a830 con 0x7f1c00128e40 2024-04-25T11:02:59.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f31827fc700 1 -- 172.21.15.155:0/1160715835 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00bcc0 con 0x7f318412f1d0 2024-04-25T11:02:59.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0008ba30 con 0x7f1c00128e40 2024-04-25T11:02:59.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c001c54e0 con 0x7f1c00128e40 2024-04-25T11:02:59.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf001a990 con 0x7f1c00128e40 2024-04-25T11:02:59.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.587+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf001ad00 con 0x7f1c00128e40 2024-04-25T11:02:59.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 -- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 msgr2=0x7f31841315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 --2- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 0x7f31841315c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f317c009e50 tx=0x7f317c00b330 comp rx=0 tx=0).stop 2024-04-25T11:02:59.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1bf00243f0 con 0x7f1c00128e40 2024-04-25T11:02:59.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f1bf005d930 con 0x7f1c00128e40 2024-04-25T11:02:59.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.589+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be00052e0 con 0x7f1c00128e40 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 -- 172.21.15.155:0/1160715835 shutdown_connections 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 --2- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f318412f1d0 0x7f31841315c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 --2- 172.21.15.155:0/1160715835 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f3184094d20 0x7f3184095100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 --2- 172.21.15.155:0/1160715835 >> v2:172.21.15.155:3300/0 conn(0x7f3184095640 0x7f3184095a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.588+0000 7f3189e51700 1 -- 172.21.15.155:0/1160715835 >> 172.21.15.155:0/1160715835 conn(0x7f318409d480 msgr2=0x7f318409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.589+0000 7f3189e51700 1 -- 172.21.15.155:0/1160715835 shutdown_connections 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.589+0000 7f3189e51700 1 -- 172.21.15.155:0/1160715835 wait complete. 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 Processor -- start 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 -- start start 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3184094d20 0x7f31841304b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 0x7f31841309f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f318412f1d0 0x7f3184130f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:02:59.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184133130 con 0x7f3184095640 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31841336d0 con 0x7f3184094d20 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3189e51700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3184133970 con 0x7f318412f1d0 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3182ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 0x7f31841309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3182ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 0x7f31841309f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40720/0 (socket says 172.21.15.155:40720) 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3182ffd700 1 -- 172.21.15.155:0/804807968 learned_addr learned my addr 172.21.15.155:0/804807968 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.590+0000 7f3183fff700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f318412f1d0 0x7f3184130f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3182ffd700 1 -- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f318412f1d0 msgr2=0x7f3184130f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f31837fe700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3184094d20 0x7f31841304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3182ffd700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f318412f1d0 0x7f3184130f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3182ffd700 1 -- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3184094d20 msgr2=0x7f31841304b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3182ffd700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3184094d20 0x7f31841304b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3182ffd700 1 -- 172.21.15.155:0/804807968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c009b30 con 0x7f3184095640 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3183fff700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f318412f1d0 0x7f3184130f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:02:59.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f31837fe700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3184094d20 0x7f31841304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3182ffd700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 0x7f31841309f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3178009fd0 tx=0x7f317800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178013610 con 0x7f3184095640 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178013770 con 0x7f3184095640 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31841328e0 con 0x7f3184095640 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178013b00 con 0x7f3184095640 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.591+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31841340f0 con 0x7f3184095640 2024-04-25T11:02:59.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.596+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f1bf0015070 con 0x7f1c00128e40 2024-04-25T11:02:59.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.592+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31841343d0 con 0x7f3184095640 2024-04-25T11:02:59.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.593+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3178013c60 con 0x7f3184095640 2024-04-25T11:02:59.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.597+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f317805aab0 con 0x7f3184095640 2024-04-25T11:02:59.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.597+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f3178039670 con 0x7f3184095640 2024-04-25T11:02:59.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.739+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa824005860 con 0x7fa84409b5d0 2024-04-25T11:02:59.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.739+0000 7fa839ffb700 1 -- 172.21.15.155:0/4195923415 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa83403a2e0 con 0x7fa84409b5d0 2024-04-25T11:02:59.740 INFO:teuthology.orchestra.run.smithi155.stdout:103079215106 2024-04-25T11:02:59.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.741+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 >> v1:172.21.15.155:6816/49624 conn(0x7fa828047620 legacy=0x7fa828049ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.741+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 msgr2=0x7fa8441c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.741+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa8441c45d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa84409d6a0 tx=0x7fa83400ebb0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 shutdown_connections 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa844132b60 0x7fa8441c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84406d4d0 0x7fa8441c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 --2- 172.21.15.155:0/4195923415 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa84409b5d0 0x7fa8441c45d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 >> 172.21.15.155:0/4195923415 conn(0x7fa84409d8b0 msgr2=0x7fa844131f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 shutdown_connections 2024-04-25T11:02:59.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.742+0000 7fa84b9e4700 1 -- 172.21.15.155:0/4195923415 wait complete. 2024-04-25T11:02:59.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215107 got 103079215106 for osd.2 2024-04-25T11:02:59.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.765+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3184132d90 con 0x7f3184095640 2024-04-25T11:02:59.765 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.765+0000 7f316bfff700 1 -- 172.21.15.155:0/804807968 <== mon.0 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3184132d90 con 0x7f3184095640 2024-04-25T11:02:59.766 INFO:teuthology.orchestra.run.smithi155.stdout:94489280514 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 >> v1:172.21.15.155:6816/49624 conn(0x7f3170047620 legacy=0x7f3170049ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 msgr2=0x7f31841309f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 0x7f31841309f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3178009fd0 tx=0x7f317800eb70 comp rx=0 tx=0).stop 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 shutdown_connections 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f318412f1d0 0x7f3184130f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3184094d20 0x7f31841304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 --2- 172.21.15.155:0/804807968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3184095640 0x7f31841309f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 >> 172.21.15.155:0/804807968 conn(0x7f318409d480 msgr2=0x7f31840708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 shutdown_connections 2024-04-25T11:02:59.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.767+0000 7f3189e51700 1 -- 172.21.15.155:0/804807968 wait complete. 2024-04-25T11:02:59.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.769+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7924005860 con 0x7f794412e770 2024-04-25T11:02:59.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.769+0000 7f792b7fe700 1 -- 172.21.15.155:0/2465454915 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7938017020 con 0x7f794412e770 2024-04-25T11:02:59.770 INFO:teuthology.orchestra.run.smithi155.stdout:98784247810 2024-04-25T11:02:59.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.771+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 >> v1:172.21.15.155:6816/49624 conn(0x7f79300475f0 legacy=0x7f7930049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 msgr2=0x7f7944090290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 0x7f7944090290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f793800c500 tx=0x7f793800c810 comp rx=0 tx=0).stop 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 shutdown_connections 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f794412f1c0 0x7f79440907d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794412b940 0x7f794408fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 --2- 172.21.15.155:0/2465454915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794412e770 0x7f7944090290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 >> 172.21.15.155:0/2465454915 conn(0x7f79440a0c70 msgr2=0x7f7944071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 shutdown_connections 2024-04-25T11:02:59.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.772+0000 7f79490f6700 1 -- 172.21.15.155:0/2465454915 wait complete. 2024-04-25T11:02:59.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280514 for osd.0 2024-04-25T11:02:59.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.1 2024-04-25T11:02:59.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.797+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1be0005590 con 0x7f1c00128e40 2024-04-25T11:02:59.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.798+0000 7f1be77fe700 1 -- 172.21.15.155:0/367909684 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bf002fa30 con 0x7f1c00128e40 2024-04-25T11:02:59.798 INFO:teuthology.orchestra.run.smithi155.stdout:107374182402 2024-04-25T11:02:59.800 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 >> v1:172.21.15.155:6816/49624 conn(0x7f1bec0475f0 legacy=0x7f1bec049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.800 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 msgr2=0x7f1c001253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 0x7f1c001253c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009840 tx=0x7f1bf0004750 comp rx=0 tx=0).stop 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 shutdown_connections 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0008c2e0 0x7f1c00125e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1c00128e40 0x7f1c001253c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 --2- 172.21.15.155:0/367909684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c00129970 0x7f1c00125900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.799+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 >> 172.21.15.155:0/367909684 conn(0x7f1c0009dac0 msgr2=0x7f1c001277c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.800+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 shutdown_connections 2024-04-25T11:02:59.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:02:59.800+0000 7f1c052d7700 1 -- 172.21.15.155:0/367909684 wait complete. 2024-04-25T11:02:59.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182403 got 107374182402 for osd.3 2024-04-25T11:03:00.756 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:03:00.781 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:03:00.786 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:03:00.812 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:03:00.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 Processor -- start 2024-04-25T11:03:00.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 -- start start 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268094d20 0x7f4268095100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4268095640 0x7f4268095a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426812f1d0 0x7f42681315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426809ce50 con 0x7f4268095640 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680974f0 con 0x7f426812f1d0 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f426e24e700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4268095fa0 con 0x7f4268094d20 2024-04-25T11:03:00.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.908+0000 7f4267fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426812f1d0 0x7f42681315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4267fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426812f1d0 0x7f42681315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40802/0 (socket says 172.21.15.155:40802) 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4267fff700 1 -- 172.21.15.155:0/873348388 learned_addr learned my addr 172.21.15.155:0/873348388 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 --2- 172.21.15.155:0/873348388 >> v2:172.21.15.155:3300/0 conn(0x7f4268095640 0x7f4268095a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f42677fe700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268094d20 0x7f4268095100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 -- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268094d20 msgr2=0x7f4268095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268094d20 0x7f4268095100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 -- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426812f1d0 msgr2=0x7f42681315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426812f1d0 0x7f42681315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 -- 172.21.15.155:0/873348388 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4268096940 con 0x7f4268095640 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f4266ffd700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4268095640 0x7f4268095a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f425c009870 tx=0x7f425c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c2ee33623e8fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f42667fc700 1 -- 172.21.15.155:0/873348388 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00f940 con 0x7f4268095640 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f42667fc700 1 -- 172.21.15.155:0/873348388 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00faa0 con 0x7f4268095640 2024-04-25T11:03:00.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.909+0000 7f42667fc700 1 -- 172.21.15.155:0/873348388 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00fe30 con 0x7f4268095640 2024-04-25T11:03:00.912 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 -- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4268095640 msgr2=0x7f4268095a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4268095640 0x7f4268095a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f425c009870 tx=0x7f425c009b80 comp rx=0 tx=0).stop 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 -- 172.21.15.155:0/873348388 shutdown_connections 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426812f1d0 0x7f42681315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268094d20 0x7f4268095100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 --2- 172.21.15.155:0/873348388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4268095640 0x7f4268095a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 -- 172.21.15.155:0/873348388 >> 172.21.15.155:0/873348388 conn(0x7f426809d480 msgr2=0x7f426809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 -- 172.21.15.155:0/873348388 shutdown_connections 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.910+0000 7f426e24e700 1 -- 172.21.15.155:0/873348388 wait complete. 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 Processor -- start 2024-04-25T11:03:00.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 -- start start 2024-04-25T11:03:00.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4268094d20 0x7f426808ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 0x7f426808f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 0x7f426808f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4268091450 con 0x7f426812f1d0 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426812d420 con 0x7f4268094d20 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.912+0000 7f426e24e700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42680922c0 con 0x7f4268095640 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4267fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 0x7f426808f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 0x7f426808f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4267fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 0x7f426808f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40728/0 (socket says 172.21.15.155:40728) 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 0x7f426808f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44562/0 (socket says 172.21.15.155:44562) 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4267fff700 1 -- 172.21.15.155:0/2845901767 learned_addr learned my addr 172.21.15.155:0/2845901767 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f42677fe700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4268094d20 0x7f426808ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.916 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 -- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4268094d20 msgr2=0x7f426808ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4268094d20 0x7f426808ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 -- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 msgr2=0x7f426808f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 0x7f426808f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 -- 172.21.15.155:0/2845901767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0094a0 con 0x7f4268095640 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4267fff700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 0x7f426808f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f42677fe700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4268094d20 0x7f426808ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.913+0000 7f4266ffd700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 0x7f426808f200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f425c018000 tx=0x7f425c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.914+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0038a0 con 0x7f4268095640 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.914+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c003a20 con 0x7f4268095640 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.914+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c003dd0 con 0x7f4268095640 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.914+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4268090a50 con 0x7f4268095640 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.914+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4268091690 con 0x7f4268095640 2024-04-25T11:03:00.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.915+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268072080 con 0x7f4268095640 2024-04-25T11:03:00.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.916+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f425c035480 con 0x7f4268095640 2024-04-25T11:03:00.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.916+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f425c013070 con 0x7f4268095640 2024-04-25T11:03:00.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.920+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f425c0071d0 con 0x7f4268095640 2024-04-25T11:03:00.941 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 Processor -- start 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 -- start start 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4095100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1fa4095640 0x7f1fa4095a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa40974f0 con 0x7f1fa412f1d0 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa409ce50 con 0x7f1fa4095640 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa9e8a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4096940 con 0x7f1fa4094d20 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa2ffd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1fa4095640 0x7f1fa4095a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa2ffd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1fa4095640 0x7f1fa4095a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40736/0 (socket says 172.21.15.155:40736) 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.941+0000 7f1fa2ffd700 1 -- 172.21.15.155:0/1779625352 learned_addr learned my addr 172.21.15.155:0/1779625352 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa37fe700 1 --2- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4095100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa2ffd700 1 -- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 msgr2=0x7f1fa4095100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa2ffd700 1 --2- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4095100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa2ffd700 1 -- 172.21.15.155:0/1779625352 >> v1:172.21.15.155:6789/0 conn(0x7f1fa412f1d0 legacy=0x7f1fa41315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa2ffd700 1 -- 172.21.15.155:0/1779625352 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4095fa0 con 0x7f1fa4095640 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa2ffd700 1 --2- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4095a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800bb90 tx=0x7f1f9800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5eb24112977ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa27fc700 1 -- 172.21.15.155:0/1779625352 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800dc20 con 0x7f1fa4095640 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa27fc700 1 -- 172.21.15.155:0/1779625352 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9800dd80 con 0x7f1fa4095640 2024-04-25T11:03:00.944 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.942+0000 7f1fa27fc700 1 -- 172.21.15.155:0/1779625352 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98017650 con 0x7f1fa4095640 2024-04-25T11:03:00.947 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 msgr2=0x7f1fa4095a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.947 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4095a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800bb90 tx=0x7f1f9800bea0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.947 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/1779625352 shutdown_connections 2024-04-25T11:03:00.947 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4095100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/1779625352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4095a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/1779625352 >> 172.21.15.155:0/1779625352 conn(0x7f1fa409d480 msgr2=0x7f1fa409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/1779625352 shutdown_connections 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.943+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/1779625352 wait complete. 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 Processor -- start 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 -- start start 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4122120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4122660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1fa412f1d0 0x7f1fa4122ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.948 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4124260 con 0x7f1fa4095640 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4092ed0 con 0x7f1fa4094d20 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.944+0000 7f1fa9e8a700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4123fb0 con 0x7f1fa412f1d0 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4122120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4122660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4122120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40834/0 (socket says 172.21.15.155:40834) 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4122660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40752/0 (socket says 172.21.15.155:40752) 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 -- 172.21.15.155:0/370979686 learned_addr learned my addr 172.21.15.155:0/370979686 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa3fff700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1fa412f1d0 0x7f1fa4122ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 -- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1fa412f1d0 msgr2=0x7f1fa4122ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1fa412f1d0 0x7f1fa4122ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 -- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 msgr2=0x7f1fa4122660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4122660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 -- 172.21.15.155:0/370979686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9800b7f0 con 0x7f1fa4094d20 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa3fff700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1fa412f1d0 0x7f1fa4122ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:00.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1fa37fe700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4122120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e9b0 tx=0x7f1f9400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.945+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400ee50 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.946+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9400c850 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.946+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa4125280 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.946+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400cbc0 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.946+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4124a20 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.947+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1f9400cd20 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.947+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa4124630 con 0x7f1fa4094d20 2024-04-25T11:03:00.950 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.947+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f1f9405c840 con 0x7f1fa4094d20 2024-04-25T11:03:00.951 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.950+0000 7f0d51457700 1 Processor -- start 2024-04-25T11:03:00.951 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.950+0000 7f0d51457700 1 -- start start 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d51457700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d51457700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d4c06e1d0 0x7f0d4c12e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d51457700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c06e630 con 0x7f0d4c12ea60 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d51457700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c06ebc0 con 0x7f0d4c06e1d0 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d51457700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c06faa0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40846/0 (socket says 172.21.15.155:40846) 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4affd700 1 -- 172.21.15.155:0/3881334071 learned_addr learned my addr 172.21.15.155:0/3881334071 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4b7fe700 1 --1- 172.21.15.155:0/3881334071 >> v1:172.21.15.155:6789/0 conn(0x7f0d4c12ea60 0x7f0d4c12eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35398/0 (socket says 172.21.15.155:35398) 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d427fc700 1 --2- 172.21.15.155:0/3881334071 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c06e1d0 0x7f0d4c12e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4affd700 1 -- 172.21.15.155:0/3881334071 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c06e1d0 msgr2=0x7f0d4c12e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4affd700 1 --2- 172.21.15.155:0/3881334071 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c06e1d0 0x7f0d4c12e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.953 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.951+0000 7f0d4a7fc700 1 -- 172.21.15.155:0/3881334071 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518001693 0 0) 0x7f0d4c06e630 con 0x7f0d4c12ea60 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f0d4affd700 1 -- 172.21.15.155:0/3881334071 >> v1:172.21.15.155:6789/0 conn(0x7f0d4c12ea60 legacy=0x7f0d4c12eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f0d4affd700 1 -- 172.21.15.155:0/3881334071 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0721b0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f0d4affd700 1 --2- 172.21.15.155:0/3881334071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c06dc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009920 tx=0x7f0d3c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b758a950de482dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f0d4a7fc700 1 -- 172.21.15.155:0/3881334071 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c011830 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f0d4a7fc700 1 -- 172.21.15.155:0/3881334071 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c011990 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f0d4a7fc700 1 -- 172.21.15.155:0/3881334071 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c011d20 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.952+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f1f9403a4f0 con 0x7f1fa4094d20 2024-04-25T11:03:00.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.953+0000 7f0d51457700 1 -- 172.21.15.155:0/3881334071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 msgr2=0x7f0d4c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.953+0000 7f0d51457700 1 --2- 172.21.15.155:0/3881334071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c06dc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009920 tx=0x7f0d3c009c30 comp rx=0 tx=0).stop 2024-04-25T11:03:00.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.955+0000 7f0d51457700 1 -- 172.21.15.155:0/3881334071 shutdown_connections 2024-04-25T11:03:00.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.955+0000 7f0d51457700 1 --2- 172.21.15.155:0/3881334071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c06dc90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.955+0000 7f0d51457700 1 --2- 172.21.15.155:0/3881334071 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c06e1d0 0x7f0d4c12e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.955+0000 7f0d51457700 1 -- 172.21.15.155:0/3881334071 >> 172.21.15.155:0/3881334071 conn(0x7f0d4c09ecb0 msgr2=0x7f0d4c09f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:00.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.956+0000 7f0d51457700 1 -- 172.21.15.155:0/3881334071 shutdown_connections 2024-04-25T11:03:00.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.956+0000 7f0d51457700 1 -- 172.21.15.155:0/3881334071 wait complete. 2024-04-25T11:03:00.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.956+0000 7f0d51457700 1 Processor -- start 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 -- start start 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c128570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06e1d0 0x7f0d4c128ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4c12ea60 0x7f0d4c1c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c129640 con 0x7f0d4c06e1d0 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c12b590 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d51457700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c12a570 con 0x7f0d4c12ea60 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c128570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c128570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:40850/0 (socket says 172.21.15.155:40850) 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 -- 172.21.15.155:0/2667375875 learned_addr learned my addr 172.21.15.155:0/2667375875 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d427fc700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06e1d0 0x7f0d4c128ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 -- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4c12ea60 msgr2=0x7f0d4c1c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4c12ea60 0x7f0d4c1c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 -- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06e1d0 msgr2=0x7f0d4c128ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.957+0000 7f0d4affd700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06e1d0 0x7f0d4c128ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d4affd700 1 -- 172.21.15.155:0/2667375875 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c009550 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d427fc700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06e1d0 0x7f0d4c128ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d4affd700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c128570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009870 tx=0x7f0d3c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c004380 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c01f070 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0036a0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c129cc0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.960 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.958+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c12adb0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.959+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0d3c021030 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.959+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f0d3c05dad0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.960+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c09cef0 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.964+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f0d3c015070 con 0x7f0d4c06d8b0 2024-04-25T11:03:00.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.983+0000 7f1144c5f700 1 Processor -- start 2024-04-25T11:03:00.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f1144c5f700 1 -- start start 2024-04-25T11:03:00.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f1144c5f700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f1140133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f1144c5f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11401341e0 0x7f11401365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f1144c5f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1140072780 con 0x7f114006e400 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f1144c5f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114006ca20 con 0x7f11401341e0 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f1144c5f700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f114006fb80 con 0x7f114006eef0 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f113e59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f114006e400 0x7f114006e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35400/0 (socket says 172.21.15.155:35400) 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f113e59c700 1 -- 172.21.15.155:0/2274290165 learned_addr learned my addr 172.21.15.155:0/2274290165 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.984+0000 7f113ed9d700 1 --2- 172.21.15.155:0/2274290165 >> v2:172.21.15.155:3300/0 conn(0x7f11401341e0 0x7f11401365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113dd9b700 1 --2- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f1140133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113ed9d700 1 -- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 msgr2=0x7f1140133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113ed9d700 1 --2- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f1140133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113ed9d700 1 -- 172.21.15.155:0/2274290165 >> v1:172.21.15.155:6789/0 conn(0x7f114006e400 legacy=0x7f114006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113ed9d700 1 -- 172.21.15.155:0/2274290165 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114006c6e0 con 0x7f11401341e0 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113ed9d700 1 --2- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401365d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1138009d90 tx=0x7f113800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc16b9fc7fcb089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113d59a700 1 -- 172.21.15.155:0/2274290165 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800e070 con 0x7f11401341e0 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113d59a700 1 -- 172.21.15.155:0/2274290165 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113800bb20 con 0x7f11401341e0 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.985+0000 7f113d59a700 1 -- 172.21.15.155:0/2274290165 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138012430 con 0x7f11401341e0 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 -- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 msgr2=0x7f11401365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 --2- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401365d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1138009d90 tx=0x7f113800b330 comp rx=0 tx=0).stop 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 -- 172.21.15.155:0/2274290165 shutdown_connections 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 --2- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401365d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 --2- 172.21.15.155:0/2274290165 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f1140133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 -- 172.21.15.155:0/2274290165 >> 172.21.15.155:0/2274290165 conn(0x7f114009dac0 msgr2=0x7f114009ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:00.989 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 -- 172.21.15.155:0/2274290165 shutdown_connections 2024-04-25T11:03:00.989 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.986+0000 7f1144c5f700 1 -- 172.21.15.155:0/2274290165 wait complete. 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 Processor -- start 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 -- start start 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f114006e400 0x7f114008fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f114008ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11400920c0 con 0x7f11401341e0 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1140133920 con 0x7f114006e400 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.987+0000 7f1144c5f700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f114008f590 con 0x7f114006eef0 2024-04-25T11:03:00.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f114008ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f114008ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:44604/0 (socket says 172.21.15.155:44604) 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 -- 172.21.15.155:0/1289742236 learned_addr learned my addr 172.21.15.155:0/1289742236 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113ed9d700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 -- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f114006e400 msgr2=0x7f114008fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f114006e400 0x7f114008fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 -- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 msgr2=0x7f11401c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 -- 172.21.15.155:0/1289742236 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009a70 con 0x7f114006eef0 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113ed9d700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f113dd9b700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f114008ff90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f113400e9e0 tx=0x7f113400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134009710 con 0x7f114006eef0 2024-04-25T11:03:00.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11400904d0 con 0x7f114006eef0 2024-04-25T11:03:00.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113400c850 con 0x7f114006eef0 2024-04-25T11:03:00.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400cbc0 con 0x7f114006eef0 2024-04-25T11:03:00.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.988+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11400914a0 con 0x7f114006eef0 2024-04-25T11:03:00.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.990+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114004f400 con 0x7f114006eef0 2024-04-25T11:03:00.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.990+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f113400cd20 con 0x7f114006eef0 2024-04-25T11:03:00.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.990+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f113405c990 con 0x7f114006eef0 2024-04-25T11:03:00.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:00.998+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f113403a5c0 con 0x7f114006eef0 2024-04-25T11:03:01.184 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.183+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4268092100 con 0x7f4268095640 2024-04-25T11:03:01.184 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.183+0000 7f424ffff700 1 -- 172.21.15.155:0/2845901767 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f425c030020 con 0x7f4268095640 2024-04-25T11:03:01.184 INFO:teuthology.orchestra.run.smithi155.stdout:103079215107 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 >> v1:172.21.15.155:6816/49624 conn(0x7f42540475f0 legacy=0x7f4254049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 msgr2=0x7f426808f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 0x7f426808f200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f425c018000 tx=0x7f425c004750 comp rx=0 tx=0).stop 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 shutdown_connections 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426812f1d0 0x7f426808f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4268094d20 0x7f426808ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 --2- 172.21.15.155:0/2845901767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4268095640 0x7f426808f200 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 >> 172.21.15.155:0/2845901767 conn(0x7f426809d480 msgr2=0x7f426812e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 shutdown_connections 2024-04-25T11:03:01.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.186+0000 7f426e24e700 1 -- 172.21.15.155:0/2845901767 wait complete. 2024-04-25T11:03:01.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215107 got 103079215107 for osd.2 2024-04-25T11:03:01.204 DEBUG:teuthology.parallel:result is None 2024-04-25T11:03:01.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.207+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1fa4067ed0 con 0x7f1fa4094d20 2024-04-25T11:03:01.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.207+0000 7f1f8bfff700 1 -- 172.21.15.155:0/370979686 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f94017030 con 0x7f1fa4094d20 2024-04-25T11:03:01.208 INFO:teuthology.orchestra.run.smithi155.stdout:98784247811 2024-04-25T11:03:01.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.209+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 >> v1:172.21.15.155:6816/49624 conn(0x7f1f900475f0 legacy=0x7f1f90049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.209+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 msgr2=0x7f1fa4122120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.209+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4122120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e9b0 tx=0x7f1f9400ecc0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 shutdown_connections 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f1fa412f1d0 0x7f1fa4122ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fa4094d20 0x7f1fa4122120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 --2- 172.21.15.155:0/370979686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fa4095640 0x7f1fa4122660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 >> 172.21.15.155:0/370979686 conn(0x7f1fa409d480 msgr2=0x7f1fa412e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 shutdown_connections 2024-04-25T11:03:01.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.210+0000 7f1fa9e8a700 1 -- 172.21.15.155:0/370979686 wait complete. 2024-04-25T11:03:01.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.1 2024-04-25T11:03:01.222 DEBUG:teuthology.parallel:result is None 2024-04-25T11:03:01.225 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.225+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0d4c06f6f0 con 0x7f0d4c06d8b0 2024-04-25T11:03:01.226 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.225+0000 7f0d43fff700 1 -- 172.21.15.155:0/2667375875 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d3c025e10 con 0x7f0d4c06d8b0 2024-04-25T11:03:01.226 INFO:teuthology.orchestra.run.smithi155.stdout:94489280516 2024-04-25T11:03:01.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 >> v1:172.21.15.155:6816/49624 conn(0x7f0d2c0475f0 legacy=0x7f0d2c049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 msgr2=0x7f0d4c128570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c128570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009870 tx=0x7f0d3c004750 comp rx=0 tx=0).stop 2024-04-25T11:03:01.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 shutdown_connections 2024-04-25T11:03:01.228 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4c12ea60 0x7f0d4c1c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c06d8b0 0x7f0d4c128570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 --2- 172.21.15.155:0/2667375875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06e1d0 0x7f0d4c128ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 >> 172.21.15.155:0/2667375875 conn(0x7f0d4c09ecb0 msgr2=0x7f0d4c09b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 shutdown_connections 2024-04-25T11:03:01.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.227+0000 7f0d51457700 1 -- 172.21.15.155:0/2667375875 wait complete. 2024-04-25T11:03:01.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.0 2024-04-25T11:03:01.240 DEBUG:teuthology.parallel:result is None 2024-04-25T11:03:01.240 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.240+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1140062510 con 0x7f114006eef0 2024-04-25T11:03:01.241 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.241+0000 7f112effd700 1 -- 172.21.15.155:0/1289742236 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1134017070 con 0x7f114006eef0 2024-04-25T11:03:01.241 INFO:teuthology.orchestra.run.smithi155.stdout:107374182403 2024-04-25T11:03:01.243 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 >> v1:172.21.15.155:6816/49624 conn(0x7f11200475f0 legacy=0x7f1120049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.243 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 msgr2=0x7f114008ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.243 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f114008ff90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f113400e9e0 tx=0x7f113400ecf0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.243 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 shutdown_connections 2024-04-25T11:03:01.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11401341e0 0x7f11401c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f114006e400 0x7f114008fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 --2- 172.21.15.155:0/1289742236 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f114006eef0 0x7f114008ff90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.242+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 >> 172.21.15.155:0/1289742236 conn(0x7f114009dac0 msgr2=0x7f11400710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.243+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 shutdown_connections 2024-04-25T11:03:01.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.243+0000 7f1144c5f700 1 -- 172.21.15.155:0/1289742236 wait complete. 2024-04-25T11:03:01.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182403 got 107374182403 for osd.3 2024-04-25T11:03:01.255 DEBUG:teuthology.parallel:result is None 2024-04-25T11:03:01.255 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:03:01.255 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 Processor -- start 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 -- start start 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e8012f6c0 0x7f5e80133e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e801343c0 0x7f5e801367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e800726b0 con 0x7f5e8012ebd0 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8006f5b0 con 0x7f5e8012f6c0 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e86953700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8006c6e0 con 0x7f5e801343c0 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7f7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e8012f6c0 0x7f5e80133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.414+0000 7f5e84ef0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e801343c0 0x7f5e801367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7ffff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5e8012ebd0 0x7f5e8012efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53468/0 (socket says 172.21.15.155:53468) 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7ffff700 1 -- 172.21.15.155:0/983036133 learned_addr learned my addr 172.21.15.155:0/983036133 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e84ef0700 1 -- 172.21.15.155:0/983036133 >> v2:172.21.15.155:3300/0 conn(0x7f5e8012f6c0 msgr2=0x7f5e80133e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e84ef0700 1 --2- 172.21.15.155:0/983036133 >> v2:172.21.15.155:3300/0 conn(0x7f5e8012f6c0 0x7f5e80133e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e84ef0700 1 -- 172.21.15.155:0/983036133 >> v1:172.21.15.155:6789/0 conn(0x7f5e8012ebd0 legacy=0x7f5e8012efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e84ef0700 1 -- 172.21.15.155:0/983036133 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8006fb70 con 0x7f5e801343c0 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7effd700 1 -- 172.21.15.155:0/983036133 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510881784 0 0) 0x7f5e800726b0 con 0x7f5e8012ebd0 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7f7fe700 1 --2- 172.21.15.155:0/983036133 >> v2:172.21.15.155:3300/0 conn(0x7f5e8012f6c0 0x7f5e80133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e84ef0700 1 --2- 172.21.15.155:0/983036133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e801343c0 0x7f5e801367b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009d90 tx=0x7f5e7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8e308ce1f5a58f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7effd700 1 -- 172.21.15.155:0/983036133 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800e070 con 0x7f5e801343c0 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7effd700 1 -- 172.21.15.155:0/983036133 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7800bb20 con 0x7f5e801343c0 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.415+0000 7f5e7effd700 1 -- 172.21.15.155:0/983036133 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78012430 con 0x7f5e801343c0 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 -- 172.21.15.155:0/983036133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e801343c0 msgr2=0x7f5e801367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.418 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 --2- 172.21.15.155:0/983036133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e801343c0 0x7f5e801367b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009d90 tx=0x7f5e7800b330 comp rx=0 tx=0).stop 2024-04-25T11:03:01.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 -- 172.21.15.155:0/983036133 shutdown_connections 2024-04-25T11:03:01.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 --2- 172.21.15.155:0/983036133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e801343c0 0x7f5e801367b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 --2- 172.21.15.155:0/983036133 >> v2:172.21.15.155:3300/0 conn(0x7f5e8012f6c0 0x7f5e80133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 -- 172.21.15.155:0/983036133 >> 172.21.15.155:0/983036133 conn(0x7f5e8009dbc0 msgr2=0x7f5e8009dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 -- 172.21.15.155:0/983036133 shutdown_connections 2024-04-25T11:03:01.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.416+0000 7f5e86953700 1 -- 172.21.15.155:0/983036133 wait complete. 2024-04-25T11:03:01.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.417+0000 7f5e86953700 1 Processor -- start 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.417+0000 7f5e86953700 1 -- start start 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.417+0000 7f5e86953700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 0x7f5e800905a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.417+0000 7f5e86953700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e8012f6c0 0x7f5e80090ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.417+0000 7f5e86953700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e801343c0 0x7f5e80091020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e86953700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80093070 con 0x7f5e8012ebd0 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e86953700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80093910 con 0x7f5e8012f6c0 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e86953700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80133c20 con 0x7f5e801343c0 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 0x7f5e800905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 0x7f5e800905a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56510/0 (socket says 172.21.15.155:56510) 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e8012f6c0 0x7f5e80090ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7ffff700 1 -- 172.21.15.155:0/128519653 learned_addr learned my addr 172.21.15.155:0/128519653 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e84ef0700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e801343c0 0x7f5e80091020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 -- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e801343c0 msgr2=0x7f5e80091020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e801343c0 0x7f5e80091020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 -- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 msgr2=0x7f5e800905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 0x7f5e800905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 -- 172.21.15.155:0/128519653 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700094a0 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7ffff700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 0x7f5e800905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7f7fe700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e8012f6c0 0x7f5e80090ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000e9b0 tx=0x7f5e7000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000ee50 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000c850 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e78009a70 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000cc00 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.418+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e801c3210 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.419+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e80070170 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.420+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5e70013470 con 0x7f5e8012f6c0 2024-04-25T11:03:01.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.420+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f5e7005cfd0 con 0x7f5e8012f6c0 2024-04-25T11:03:01.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.424+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f5e7003ad60 con 0x7f5e8012f6c0 2024-04-25T11:03:01.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.578+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e8006fe10 con 0x7f5e6c0475f0 2024-04-25T11:03:01.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.580+0000 7f5e7cff9700 1 -- 172.21.15.155:0/128519653 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34625 (unknown 2965378022 0 3847683578) 0x7f5e74001760 con 0x7f5e6c0475f0 2024-04-25T11:03:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:01.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.581+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 >> v1:172.21.15.155:6816/49624 conn(0x7f5e6c0475f0 legacy=0x7f5e6c049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.581+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e8012f6c0 msgr2=0x7f5e80090ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.581+0000 7f5e86953700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e8012f6c0 0x7f5e80090ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000e9b0 tx=0x7f5e7000ecc0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 shutdown_connections 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5e801343c0 0x7f5e80091020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e8012ebd0 0x7f5e800905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 --2- 172.21.15.155:0/128519653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e8012f6c0 0x7f5e80090ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 >> 172.21.15.155:0/128519653 conn(0x7f5e8009dbc0 msgr2=0x7f5e800703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 shutdown_connections 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.582+0000 7f5e86953700 1 -- 172.21.15.155:0/128519653 wait complete. 2024-04-25T11:03:01.583 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T11:02:59.992111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110980,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376380,"statfs":{"total":386547056640,"available":386433413120,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":327696,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":65,"num_write":90,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000693"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224548+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:57.224548+0000","last_peered":"2024-04-25T11:02:57.224548+0000","last_clean":"2024-04-25T11:02:57.224548+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:57.224548+0000","last_undegraded":"2024-04-25T11:02:57.224548+0000","last_fullsized":"2024-04-25T11:02:57.224548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:33.129142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226911+0000","last_change":"2024-04-25T11:02:52.039896+0000","last_active":"2024-04-25T11:02:57.226911+0000","last_peered":"2024-04-25T11:02:57.226911+0000","last_clean":"2024-04-25T11:02:57.226911+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:02:57.226911+0000","last_undegraded":"2024-04-25T11:02:57.226911+0000","last_fullsized":"2024-04-25T11:02:57.226911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.972806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224526+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:57.224526+0000","last_peered":"2024-04-25T11:02:57.224526+0000","last_clean":"2024-04-25T11:02:57.224526+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:57.224526+0000","last_undegraded":"2024-04-25T11:02:57.224526+0000","last_fullsized":"2024-04-25T11:02:57.224526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:24.325806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:23.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":157,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226905+0000","last_change":"2024-04-25T11:02:52.039856+0000","last_active":"2024-04-25T11:02:57.226905+0000","last_peered":"2024-04-25T11:02:57.226905+0000","last_clean":"2024-04-25T11:02:57.226905+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:02:57.226905+0000","last_undegraded":"2024-04-25T11:02:57.226905+0000","last_fullsized":"2024-04-25T11:02:57.226905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224458+0000","last_change":"2024-04-25T11:02:54.049124+0000","last_active":"2024-04-25T11:02:57.224458+0000","last_peered":"2024-04-25T11:02:57.224458+0000","last_clean":"2024-04-25T11:02:57.224458+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:02:57.224458+0000","last_undegraded":"2024-04-25T11:02:57.224458+0000","last_fullsized":"2024-04-25T11:02:57.224458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:18.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T11:02:56.752416+0000","last_change":"2024-04-25T11:02:54.048580+0000","last_active":"2024-04-25T11:02:56.752416+0000","last_peered":"2024-04-25T11:02:56.752416+0000","last_clean":"2024-04-25T11:02:56.752416+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:02:56.752416+0000","last_undegraded":"2024-04-25T11:02:56.752416+0000","last_fullsized":"2024-04-25T11:02:56.752416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:35.802506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.268917+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.092289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226923+0000","last_change":"2024-04-25T11:02:52.040177+0000","last_active":"2024-04-25T11:02:57.226923+0000","last_peered":"2024-04-25T11:02:57.226923+0000","last_clean":"2024-04-25T11:02:57.226923+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:02:57.226923+0000","last_undegraded":"2024-04-25T11:02:57.226923+0000","last_fullsized":"2024-04-25T11:02:57.226923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:54.647842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182403,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]}]},{"osd":2,"up_from":24,"seq":103079215107,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]}]},{"osd":0,"up_from":22,"seq":94489280516,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":23,"seq":98784247811,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:01.595 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 Processor -- start 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 -- start start 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 0x7fb6bc12f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 0x7fb6bc06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 0x7fb6bc06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6bc06fb70 con 0x7fb6bc06dac0 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc072440 con 0x7fb6bc12f6f0 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc06fd80 con 0x7fb6bc12edd0 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6ba7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 0x7fb6bc06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6bb7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 0x7fb6bc06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 0x7fb6bc12f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 0x7fb6bc12f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52380/0 (socket says 172.21.15.155:52380) 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6ba7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 0x7fb6bc06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49458/0 (socket says 172.21.15.155:49458) 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6bb7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 0x7fb6bc06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56524/0 (socket says 172.21.15.155:56524) 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 -- 172.21.15.155:0/3758538776 learned_addr learned my addr 172.21.15.155:0/3758538776 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 -- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 msgr2=0x7fb6bc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 --2- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 0x7fb6bc06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 -- 172.21.15.155:0/3758538776 >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 msgr2=0x7fb6bc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 --2- 172.21.15.155:0/3758538776 >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 0x7fb6bc06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 -- 172.21.15.155:0/3758538776 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc06ffb0 con 0x7fb6bc12edd0 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6ba7fc700 1 --2- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 0x7fb6bc06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6bb7fe700 1 --2- 172.21.15.155:0/3758538776 >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 0x7fb6bc06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6baffd700 1 --2- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 0x7fb6bc12f1b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0098f0 tx=0x7fb6ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e42e5b83af9599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6b9ffb700 1 -- 172.21.15.155:0/3758538776 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac011830 con 0x7fb6bc12edd0 2024-04-25T11:03:01.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6b9ffb700 1 -- 172.21.15.155:0/3758538776 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac011990 con 0x7fb6bc12edd0 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6b9ffb700 1 -- 172.21.15.155:0/3758538776 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac011d20 con 0x7fb6bc12edd0 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 -- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 msgr2=0x7fb6bc12f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.714+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 0x7fb6bc12f1b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0098f0 tx=0x7fb6ac009c00 comp rx=0 tx=0).stop 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 -- 172.21.15.155:0/3758538776 shutdown_connections 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3758538776 >> v2:172.21.15.155:3300/0 conn(0x7fb6bc06dac0 0x7fb6bc06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12edd0 0x7fb6bc12f1b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3758538776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc12f6f0 0x7fb6bc06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 -- 172.21.15.155:0/3758538776 >> 172.21.15.155:0/3758538776 conn(0x7fb6bc09e9f0 msgr2=0x7fb6bc09ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 -- 172.21.15.155:0/3758538776 shutdown_connections 2024-04-25T11:03:01.718 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.715+0000 7fb6c1480700 1 -- 172.21.15.155:0/3758538776 wait complete. 2024-04-25T11:03:01.720 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 Processor -- start 2024-04-25T11:03:01.720 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 -- start start 2024-04-25T11:03:01.720 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 0x7fb6bc0957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6bc12edd0 0x7fb6bc095ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12f6f0 0x7fb6bc096220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc124220 con 0x7fb6bc12edd0 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc123f10 con 0x7fb6bc06dac0 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.716+0000 7fb6c1480700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc098180 con 0x7fb6bc12f6f0 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 0x7fb6bc0957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6bb7fe700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12f6f0 0x7fb6bc096220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 0x7fb6bc0957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49464/0 (socket says 172.21.15.155:49464) 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 -- 172.21.15.155:0/3243446081 learned_addr learned my addr 172.21.15.155:0/3243446081 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6ba7fc700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6bc12edd0 0x7fb6bc095ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 -- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12f6f0 msgr2=0x7fb6bc096220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12f6f0 0x7fb6bc096220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 -- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6bc12edd0 msgr2=0x7fb6bc095ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6bc12edd0 0x7fb6bc095ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.721 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 -- 172.21.15.155:0/3243446081 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009a70 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6baffd700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 0x7fb6bc0957a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac000c00 tx=0x7fb6ac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac004450 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac0045b0 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b00094a0 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0038c0 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.717+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac009550 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.718+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fb6ac021070 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.718+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc062510 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.718+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fb6ac05d990 con 0x7fb6bc06dac0 2024-04-25T11:03:01.722 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.722+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fb6ac026a50 con 0x7fb6bc06dac0 2024-04-25T11:03:01.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.879+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6bc06f400 con 0x7fb6a80475f0 2024-04-25T11:03:01.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.880+0000 7fb6a37fe700 1 -- 172.21.15.155:0/3243446081 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34625 (unknown 2965378022 0 3847683578) 0x7fb6b0009860 con 0x7fb6a80475f0 2024-04-25T11:03:01.881 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:01.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 >> v1:172.21.15.155:6816/49624 conn(0x7fb6a80475f0 legacy=0x7fb6a8049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 msgr2=0x7fb6bc0957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:01.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 0x7fb6bc0957a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac000c00 tx=0x7fb6ac004750 comp rx=0 tx=0).stop 2024-04-25T11:03:01.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 shutdown_connections 2024-04-25T11:03:01.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fb6bc12f6f0 0x7fb6bc096220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6bc06dac0 0x7fb6bc0957a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 --2- 172.21.15.155:0/3243446081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6bc12edd0 0x7fb6bc095ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:01.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 >> 172.21.15.155:0/3243446081 conn(0x7fb6bc09e9f0 msgr2=0x7fb6bc070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:01.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 shutdown_connections 2024-04-25T11:03:01.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:01.882+0000 7fb6c1480700 1 -- 172.21.15.155:0/3243446081 wait complete. 2024-04-25T11:03:01.884 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:01.895 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T11:02:59.992111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110980,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376380,"statfs":{"total":386547056640,"available":386433413120,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":327696,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":65,"num_write":90,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000693"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224548+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:57.224548+0000","last_peered":"2024-04-25T11:02:57.224548+0000","last_clean":"2024-04-25T11:02:57.224548+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:57.224548+0000","last_undegraded":"2024-04-25T11:02:57.224548+0000","last_fullsized":"2024-04-25T11:02:57.224548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:33.129142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226911+0000","last_change":"2024-04-25T11:02:52.039896+0000","last_active":"2024-04-25T11:02:57.226911+0000","last_peered":"2024-04-25T11:02:57.226911+0000","last_clean":"2024-04-25T11:02:57.226911+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:02:57.226911+0000","last_undegraded":"2024-04-25T11:02:57.226911+0000","last_fullsized":"2024-04-25T11:02:57.226911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.972806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224526+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:57.224526+0000","last_peered":"2024-04-25T11:02:57.224526+0000","last_clean":"2024-04-25T11:02:57.224526+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:57.224526+0000","last_undegraded":"2024-04-25T11:02:57.224526+0000","last_fullsized":"2024-04-25T11:02:57.224526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:24.325806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:23.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":157,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226905+0000","last_change":"2024-04-25T11:02:52.039856+0000","last_active":"2024-04-25T11:02:57.226905+0000","last_peered":"2024-04-25T11:02:57.226905+0000","last_clean":"2024-04-25T11:02:57.226905+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:02:57.226905+0000","last_undegraded":"2024-04-25T11:02:57.226905+0000","last_fullsized":"2024-04-25T11:02:57.226905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224458+0000","last_change":"2024-04-25T11:02:54.049124+0000","last_active":"2024-04-25T11:02:57.224458+0000","last_peered":"2024-04-25T11:02:57.224458+0000","last_clean":"2024-04-25T11:02:57.224458+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:02:57.224458+0000","last_undegraded":"2024-04-25T11:02:57.224458+0000","last_fullsized":"2024-04-25T11:02:57.224458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:18.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T11:02:56.752416+0000","last_change":"2024-04-25T11:02:54.048580+0000","last_active":"2024-04-25T11:02:56.752416+0000","last_peered":"2024-04-25T11:02:56.752416+0000","last_clean":"2024-04-25T11:02:56.752416+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:02:56.752416+0000","last_undegraded":"2024-04-25T11:02:56.752416+0000","last_fullsized":"2024-04-25T11:02:56.752416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:35.802506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.268917+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.092289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226923+0000","last_change":"2024-04-25T11:02:52.040177+0000","last_active":"2024-04-25T11:02:57.226923+0000","last_peered":"2024-04-25T11:02:57.226923+0000","last_clean":"2024-04-25T11:02:57.226923+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:02:57.226923+0000","last_undegraded":"2024-04-25T11:02:57.226923+0000","last_fullsized":"2024-04-25T11:02:57.226923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:54.647842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182403,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]}]},{"osd":2,"up_from":24,"seq":103079215107,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]}]},{"osd":0,"up_from":22,"seq":94489280516,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":23,"seq":98784247811,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:01.896 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:03:01.896 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:03:01.896 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:03:01.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 Processor -- start 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 -- start start 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc84812ed10 0x7fc84812f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc848136a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc848093940 con 0x7fc848072000 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84806d360 con 0x7fc848072b90 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc84806ab70 con 0x7fc84812ed10 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84c81d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc848072000 0x7fc848072480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53478/0 (socket says 172.21.15.155:53478) 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84c81d700 1 -- 172.21.15.155:0/229332115 learned_addr learned my addr 172.21.15.155:0/229332115 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d01e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc84812ed10 0x7fc84812f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.018 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d81f700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc848136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc83ffff700 1 -- 172.21.15.155:0/229332115 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821183231 0 0) 0x7fc848093940 con 0x7fc848072000 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d01e700 1 -- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 msgr2=0x7fc848136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d01e700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc848136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d01e700 1 -- 172.21.15.155:0/229332115 >> v1:172.21.15.155:6789/0 conn(0x7fc848072000 legacy=0x7fc848072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d01e700 1 -- 172.21.15.155:0/229332115 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848093d90 con 0x7fc84812ed10 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d81f700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc848136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84d01e700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc84812ed10 0x7fc84812f0f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc844009870 tx=0x7fc844009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349939e79cececab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc83ffff700 1 -- 172.21.15.155:0/229332115 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844011830 con 0x7fc84812ed10 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc83ffff700 1 -- 172.21.15.155:0/229332115 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc844011990 con 0x7fc84812ed10 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc83ffff700 1 -- 172.21.15.155:0/229332115 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844011d20 con 0x7fc84812ed10 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 -- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc84812ed10 msgr2=0x7fc84812f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc84812ed10 0x7fc84812f0f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc844009870 tx=0x7fc844009b80 comp rx=0 tx=0).stop 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 -- 172.21.15.155:0/229332115 shutdown_connections 2024-04-25T11:03:02.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc84812ed10 0x7fc84812f0f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.016+0000 7fc84f282700 1 --2- 172.21.15.155:0/229332115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc848136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.017+0000 7fc84f282700 1 -- 172.21.15.155:0/229332115 >> 172.21.15.155:0/229332115 conn(0x7fc84809a140 msgr2=0x7fc84806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:02.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.017+0000 7fc84f282700 1 -- 172.21.15.155:0/229332115 shutdown_connections 2024-04-25T11:03:02.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.017+0000 7fc84f282700 1 -- 172.21.15.155:0/229332115 wait complete. 2024-04-25T11:03:02.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.017+0000 7fc84f282700 1 Processor -- start 2024-04-25T11:03:02.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.017+0000 7fc84f282700 1 -- start start 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc848072000 0x7fc8481243b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc8481248f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc84812ed10 0x7fc848124e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848125e30 con 0x7fc84812ed10 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc848128b50 con 0x7fc848072b90 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc848128e60 con 0x7fc848072000 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84d01e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc848072000 0x7fc8481243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc8481248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84d81f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc84812ed10 0x7fc848124e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84d01e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc848072000 0x7fc8481243b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52398/0 (socket says 172.21.15.155:52398) 2024-04-25T11:03:02.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84d01e700 1 -- 172.21.15.155:0/237536357 learned_addr learned my addr 172.21.15.155:0/237536357 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 -- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc848072000 msgr2=0x7fc8481243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc848072000 0x7fc8481243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 -- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc84812ed10 msgr2=0x7fc848124e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc84812ed10 0x7fc848124e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 -- 172.21.15.155:0/237536357 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840009a70 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84d81f700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc84812ed10 0x7fc848124e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84c81d700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc8481248f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc84806ae20 tx=0x7fc83800c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838004c80 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838004de0 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8440094a0 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.018+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838007310 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.019+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8481c33b0 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.019+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fc838007470 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.019+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc848091ff0 con 0x7fc848072b90 2024-04-25T11:03:02.023 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.020+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fc83805c360 con 0x7fc848072b90 2024-04-25T11:03:02.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.024+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fc838039e40 con 0x7fc848072b90 2024-04-25T11:03:02.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.216+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc848092220 con 0x7fc848072b90 2024-04-25T11:03:02.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.217+0000 7fc83dffb700 1 -- 172.21.15.155:0/237536357 <== mon.1 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc838007720 con 0x7fc848072b90 2024-04-25T11:03:02.218 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:02.218 INFO:teuthology.orchestra.run.smithi155.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:03:02.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.218+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 >> v1:172.21.15.155:6816/49624 conn(0x7fc82c0475f0 legacy=0x7fc82c049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.218+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 msgr2=0x7fc8481248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.218+0000 7fc84f282700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc8481248f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc84806ae20 tx=0x7fc83800c750 comp rx=0 tx=0).stop 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 shutdown_connections 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc84812ed10 0x7fc848124e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fc848072000 0x7fc8481243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 --2- 172.21.15.155:0/237536357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc848072b90 0x7fc8481248f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 >> 172.21.15.155:0/237536357 conn(0x7fc84809a140 msgr2=0x7fc8480928a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 shutdown_connections 2024-04-25T11:03:02.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.219+0000 7fc84f282700 1 -- 172.21.15.155:0/237536357 wait complete. 2024-04-25T11:03:02.231 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:03:02.231 INFO:teuthology.run_tasks:Running task print... 2024-04-25T11:03:02.274 INFO:teuthology.task.print:**** done ceph.restart 2024-04-25T11:03:02.274 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-04-25T11:03:02.283 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-04-25T11:03:02.292 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2024-04-25T11:03:02.301 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T11:03:02.310 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:03:02.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:02.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.429+0000 7f81b4582700 1 Processor -- start 2024-04-25T11:03:02.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.429+0000 7f81b4582700 1 -- start start 2024-04-25T11:03:02.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b4582700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 0x7f81ac12b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b4582700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81ac128840 0x7f81ac133a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b4582700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac06d440 con 0x7f81ac12bd70 2024-04-25T11:03:02.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b4582700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac072f50 con 0x7f81ac128840 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b4582700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac06cb40 con 0x7f81ac12b450 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b231e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 0x7f81ac12b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81ac128840 0x7f81ac133a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b231e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 0x7f81ac12b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49504/0 (socket says 172.21.15.155:49504) 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81ac128840 0x7f81ac133a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56554/0 (socket says 172.21.15.155:56554) 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b231e700 1 -- 172.21.15.155:0/707179565 learned_addr learned my addr 172.21.15.155:0/707179565 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b1b1d700 1 --1- 172.21.15.155:0/707179565 >> v1:172.21.15.155:6789/0 conn(0x7f81ac12bd70 0x7f81ac128130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53486/0 (socket says 172.21.15.155:53486) 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 -- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 msgr2=0x7f81ac12b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 --2- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 0x7f81ac12b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 -- 172.21.15.155:0/707179565 >> v1:172.21.15.155:6789/0 conn(0x7f81ac12bd70 legacy=0x7f81ac128130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 -- 172.21.15.155:0/707179565 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac06ab70 con 0x7f81ac128840 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b131c700 1 -- 172.21.15.155:0/707179565 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420515668 0 0) 0x7f81ac06d440 con 0x7f81ac12bd70 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b231e700 1 --2- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 0x7f81ac12b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:02.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.430+0000 7f81b2b1f700 1 --2- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac128840 0x7f81ac133a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f81a400f370 tx=0x7f81a400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faff98efd928d8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b131c700 1 -- 172.21.15.155:0/707179565 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4012070 con 0x7f81ac128840 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b131c700 1 -- 172.21.15.155:0/707179565 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400fe70 con 0x7f81ac128840 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b131c700 1 -- 172.21.15.155:0/707179565 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400d2f0 con 0x7f81ac128840 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 -- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac128840 msgr2=0x7f81ac133a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 --2- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac128840 0x7f81ac133a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f81a400f370 tx=0x7f81a400f680 comp rx=0 tx=0).stop 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 -- 172.21.15.155:0/707179565 shutdown_connections 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 --2- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac128840 0x7f81ac133a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 --2- 172.21.15.155:0/707179565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac12b450 0x7f81ac12b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 -- 172.21.15.155:0/707179565 >> 172.21.15.155:0/707179565 conn(0x7f81ac09a140 msgr2=0x7f81ac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 -- 172.21.15.155:0/707179565 shutdown_connections 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.431+0000 7f81b4582700 1 -- 172.21.15.155:0/707179565 wait complete. 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 Processor -- start 2024-04-25T11:03:02.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 -- start start 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 0x7f81ac08b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81ac12b450 0x7f81ac08b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 0x7f81ac08ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac08ca80 con 0x7f81ac12bd70 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac090d20 con 0x7f81ac128840 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b4582700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac08dd80 con 0x7f81ac12b450 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b231e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 0x7f81ac08b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b2b1f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 0x7f81ac08ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b2b1f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 0x7f81ac08ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56556/0 (socket says 172.21.15.155:56556) 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b2b1f700 1 -- 172.21.15.155:0/3184919976 learned_addr learned my addr 172.21.15.155:0/3184919976 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.432+0000 7f81b231e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 0x7f81ac08b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49512/0 (socket says 172.21.15.155:49512) 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b1b1d700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81ac12b450 0x7f81ac08b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b2b1f700 1 -- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81ac12b450 msgr2=0x7f81ac08b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b2b1f700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81ac12b450 0x7f81ac08b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b2b1f700 1 -- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 msgr2=0x7f81ac08b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b2b1f700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 0x7f81ac08b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b2b1f700 1 -- 172.21.15.155:0/3184919976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c009550 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b231e700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 0x7f81ac08b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b2b1f700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 0x7f81ac08ba80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f81a400f9d0 tx=0x7f81a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4020640 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400d2e0 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a400f050 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40209e0 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.433+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac08d9b0 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.434+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f81a4020b40 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.434+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac128c80 con 0x7f81ac12bd70 2024-04-25T11:03:02.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.434+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f81a4062300 con 0x7f81ac12bd70 2024-04-25T11:03:02.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.438+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f81ac128c80 con 0x7f81ac12bd70 2024-04-25T11:03:02.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.595+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81ac12c190 con 0x7f81900475f0 2024-04-25T11:03:02.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.596+0000 7f81a2ffd700 1 -- 172.21.15.155:0/3184919976 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34599 (unknown 2965378022 0 1829076952) 0x7f819c009c10 con 0x7f81900475f0 2024-04-25T11:03:02.596 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 >> v1:172.21.15.155:6816/49624 conn(0x7f81900475f0 legacy=0x7f8190049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 msgr2=0x7f81ac08ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 0x7f81ac08ba80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f81a400f9d0 tx=0x7f81a4004ad0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 shutdown_connections 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81ac12bd70 0x7f81ac08ba80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81ac128840 0x7f81ac08b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 --2- 172.21.15.155:0/3184919976 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f81ac12b450 0x7f81ac08b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 >> 172.21.15.155:0/3184919976 conn(0x7f81ac09a140 msgr2=0x7f81ac129be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:02.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 shutdown_connections 2024-04-25T11:03:02.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.598+0000 7f81b4582700 1 -- 172.21.15.155:0/3184919976 wait complete. 2024-04-25T11:03:02.599 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:02.609 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-25T11:03:01.992534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110980,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376380,"statfs":{"total":386547056640,"available":386433413120,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":327696,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":65,"num_write":90,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001116"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224548+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:57.224548+0000","last_peered":"2024-04-25T11:02:57.224548+0000","last_clean":"2024-04-25T11:02:57.224548+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:57.224548+0000","last_undegraded":"2024-04-25T11:02:57.224548+0000","last_fullsized":"2024-04-25T11:02:57.224548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:33.129142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226911+0000","last_change":"2024-04-25T11:02:52.039896+0000","last_active":"2024-04-25T11:02:57.226911+0000","last_peered":"2024-04-25T11:02:57.226911+0000","last_clean":"2024-04-25T11:02:57.226911+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:02:57.226911+0000","last_undegraded":"2024-04-25T11:02:57.226911+0000","last_fullsized":"2024-04-25T11:02:57.226911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.972806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224526+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:57.224526+0000","last_peered":"2024-04-25T11:02:57.224526+0000","last_clean":"2024-04-25T11:02:57.224526+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:57.224526+0000","last_undegraded":"2024-04-25T11:02:57.224526+0000","last_fullsized":"2024-04-25T11:02:57.224526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:24.325806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:23.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":157,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226905+0000","last_change":"2024-04-25T11:02:52.039856+0000","last_active":"2024-04-25T11:02:57.226905+0000","last_peered":"2024-04-25T11:02:57.226905+0000","last_clean":"2024-04-25T11:02:57.226905+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:02:57.226905+0000","last_undegraded":"2024-04-25T11:02:57.226905+0000","last_fullsized":"2024-04-25T11:02:57.226905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224458+0000","last_change":"2024-04-25T11:02:54.049124+0000","last_active":"2024-04-25T11:02:57.224458+0000","last_peered":"2024-04-25T11:02:57.224458+0000","last_clean":"2024-04-25T11:02:57.224458+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:02:57.224458+0000","last_undegraded":"2024-04-25T11:02:57.224458+0000","last_fullsized":"2024-04-25T11:02:57.224458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:18.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T11:02:56.752416+0000","last_change":"2024-04-25T11:02:54.048580+0000","last_active":"2024-04-25T11:02:56.752416+0000","last_peered":"2024-04-25T11:02:56.752416+0000","last_clean":"2024-04-25T11:02:56.752416+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:02:56.752416+0000","last_undegraded":"2024-04-25T11:02:56.752416+0000","last_fullsized":"2024-04-25T11:02:56.752416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:35.802506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.268917+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.092289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226923+0000","last_change":"2024-04-25T11:02:52.040177+0000","last_active":"2024-04-25T11:02:57.226923+0000","last_peered":"2024-04-25T11:02:57.226923+0000","last_clean":"2024-04-25T11:02:57.226923+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:02:57.226923+0000","last_undegraded":"2024-04-25T11:02:57.226923+0000","last_fullsized":"2024-04-25T11:02:57.226923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:54.647842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182403,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]}]},{"osd":2,"up_from":24,"seq":103079215108,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":0,"up_from":22,"seq":94489280516,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":23,"seq":98784247812,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:02.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:02.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 Processor -- start 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 -- start start 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa4b0095ad0 0x7fa4b0095eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b00963f0 0x7fa4b0093790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b00725b0 con 0x7fa4b0097000 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b009be10 con 0x7fa4b0095ad0 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b8533700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b006ab70 con 0x7fa4b00963f0 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b6ad0700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b00963f0 0x7fa4b0093790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.730+0000 7fa4b62cf700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa4b0095ad0 0x7fa4b0095eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b5ace700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa4b0097000 0x7fa4b0093080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53502/0 (socket says 172.21.15.155:53502) 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b5ace700 1 -- 172.21.15.155:0/111776247 learned_addr learned my addr 172.21.15.155:0/111776247 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:02.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b62cf700 1 -- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b00963f0 msgr2=0x7fa4b0093790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b62cf700 1 --2- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b00963f0 0x7fa4b0093790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b62cf700 1 -- 172.21.15.155:0/111776247 >> v1:172.21.15.155:6789/0 conn(0x7fa4b0097000 legacy=0x7fa4b0093080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b62cf700 1 -- 172.21.15.155:0/111776247 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b006dd30 con 0x7fa4b0095ad0 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b52cd700 1 -- 172.21.15.155:0/111776247 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996649870 0 0) 0x7fa4b00725b0 con 0x7fa4b0097000 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b6ad0700 1 --2- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b00963f0 0x7fa4b0093790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b62cf700 1 --2- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b0095ad0 0x7fa4b0095eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00bae0 tx=0x7fa4ac00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b83129006636b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b52cd700 1 -- 172.21.15.155:0/111776247 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00dc70 con 0x7fa4b0095ad0 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b52cd700 1 -- 172.21.15.155:0/111776247 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac00ddd0 con 0x7fa4b0095ad0 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.731+0000 7fa4b52cd700 1 -- 172.21.15.155:0/111776247 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac010690 con 0x7fa4b0095ad0 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 -- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b0095ad0 msgr2=0x7fa4b0095eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 --2- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b0095ad0 0x7fa4b0095eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00bae0 tx=0x7fa4ac00bdf0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 -- 172.21.15.155:0/111776247 shutdown_connections 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 --2- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b00963f0 0x7fa4b0093790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 --2- 172.21.15.155:0/111776247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b0095ad0 0x7fa4b0095eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 -- 172.21.15.155:0/111776247 >> 172.21.15.155:0/111776247 conn(0x7fa4b009d480 msgr2=0x7fa4b009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 -- 172.21.15.155:0/111776247 shutdown_connections 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 -- 172.21.15.155:0/111776247 wait complete. 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.732+0000 7fa4b8533700 1 Processor -- start 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 -- start start 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 0x7fa4b0128120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00963f0 0x7fa4b0128660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b0097000 0x7fa4b0128ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01258c0 con 0x7fa4b00963f0 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01278e0 con 0x7fa4b0097000 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b8533700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0126840 con 0x7fa4b0095ad0 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 0x7fa4b0128120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b5ace700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00963f0 0x7fa4b0128660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 0x7fa4b0128120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52418/0 (socket says 172.21.15.155:52418) 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 -- 172.21.15.155:0/1180497767 learned_addr learned my addr 172.21.15.155:0/1180497767 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b6ad0700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b0097000 0x7fa4b0128ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 -- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b0097000 msgr2=0x7fa4b0128ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b0097000 0x7fa4b0128ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 -- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00963f0 msgr2=0x7fa4b0128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00963f0 0x7fa4b0128660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 -- 172.21.15.155:0/1180497767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a8009a70 con 0x7fa4b0095ad0 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b5ace700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00963f0 0x7fa4b0128660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.733+0000 7fa4b62cf700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 0x7fa4b0128120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00b2c0 tx=0x7fa4ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.734+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac003ab0 con 0x7fa4b0095ad0 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.734+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac003c30 con 0x7fa4b0095ad0 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.734+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac00b740 con 0x7fa4b0095ad0 2024-04-25T11:03:02.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.734+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac010470 con 0x7fa4b0095ad0 2024-04-25T11:03:02.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.734+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b01bea40 con 0x7fa4b0095ad0 2024-04-25T11:03:02.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.734+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b0062510 con 0x7fa4b0095ad0 2024-04-25T11:03:02.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.735+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa4ac019400 con 0x7fa4b0095ad0 2024-04-25T11:03:02.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.735+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fa4ac05d900 con 0x7fa4b0095ad0 2024-04-25T11:03:02.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.739+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fa4ac02cb10 con 0x7fa4b0095ad0 2024-04-25T11:03:02.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.893+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4b00949c0 con 0x7fa4940475f0 2024-04-25T11:03:02.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.895+0000 7fa4a6ffd700 1 -- 172.21.15.155:0/1180497767 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34599 (unknown 2965378022 0 1829076952) 0x7fa4a000ea30 con 0x7fa4940475f0 2024-04-25T11:03:02.895 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 >> v1:172.21.15.155:6816/49624 conn(0x7fa4940475f0 legacy=0x7fa494049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 msgr2=0x7fa4b0128120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 0x7fa4b0128120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00b2c0 tx=0x7fa4ac004750 comp rx=0 tx=0).stop 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 shutdown_connections 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b0097000 0x7fa4b0128ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa4b0095ad0 0x7fa4b0128120 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 --2- 172.21.15.155:0/1180497767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00963f0 0x7fa4b0128660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:02.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 >> 172.21.15.155:0/1180497767 conn(0x7fa4b009d480 msgr2=0x7fa4b0093290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:02.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 shutdown_connections 2024-04-25T11:03:02.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:02.896+0000 7fa4b8533700 1 -- 172.21.15.155:0/1180497767 wait complete. 2024-04-25T11:03:02.897 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:02.907 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-25T11:03:01.992534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110980,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376380,"statfs":{"total":386547056640,"available":386433413120,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":327696,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":65,"num_write":90,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001116"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224548+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:57.224548+0000","last_peered":"2024-04-25T11:02:57.224548+0000","last_clean":"2024-04-25T11:02:57.224548+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:57.224548+0000","last_undegraded":"2024-04-25T11:02:57.224548+0000","last_fullsized":"2024-04-25T11:02:57.224548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:33.129142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226911+0000","last_change":"2024-04-25T11:02:52.039896+0000","last_active":"2024-04-25T11:02:57.226911+0000","last_peered":"2024-04-25T11:02:57.226911+0000","last_clean":"2024-04-25T11:02:57.226911+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:02:57.226911+0000","last_undegraded":"2024-04-25T11:02:57.226911+0000","last_fullsized":"2024-04-25T11:02:57.226911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.972806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224526+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:57.224526+0000","last_peered":"2024-04-25T11:02:57.224526+0000","last_clean":"2024-04-25T11:02:57.224526+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:57.224526+0000","last_undegraded":"2024-04-25T11:02:57.224526+0000","last_fullsized":"2024-04-25T11:02:57.224526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:24.325806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:23.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":157,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226905+0000","last_change":"2024-04-25T11:02:52.039856+0000","last_active":"2024-04-25T11:02:57.226905+0000","last_peered":"2024-04-25T11:02:57.226905+0000","last_clean":"2024-04-25T11:02:57.226905+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:02:57.226905+0000","last_undegraded":"2024-04-25T11:02:57.226905+0000","last_fullsized":"2024-04-25T11:02:57.226905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224458+0000","last_change":"2024-04-25T11:02:54.049124+0000","last_active":"2024-04-25T11:02:57.224458+0000","last_peered":"2024-04-25T11:02:57.224458+0000","last_clean":"2024-04-25T11:02:57.224458+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:02:57.224458+0000","last_undegraded":"2024-04-25T11:02:57.224458+0000","last_fullsized":"2024-04-25T11:02:57.224458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:18.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T11:02:56.752416+0000","last_change":"2024-04-25T11:02:54.048580+0000","last_active":"2024-04-25T11:02:56.752416+0000","last_peered":"2024-04-25T11:02:56.752416+0000","last_clean":"2024-04-25T11:02:56.752416+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:02:56.752416+0000","last_undegraded":"2024-04-25T11:02:56.752416+0000","last_fullsized":"2024-04-25T11:02:56.752416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:35.802506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.268917+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.092289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226923+0000","last_change":"2024-04-25T11:02:52.040177+0000","last_active":"2024-04-25T11:02:57.226923+0000","last_peered":"2024-04-25T11:02:57.226923+0000","last_clean":"2024-04-25T11:02:57.226923+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:02:57.226923+0000","last_undegraded":"2024-04-25T11:02:57.226923+0000","last_fullsized":"2024-04-25T11:02:57.226923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:54.647842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182403,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]}]},{"osd":2,"up_from":24,"seq":103079215108,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":0,"up_from":22,"seq":94489280516,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":23,"seq":98784247812,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:02.908 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:03:02.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:03.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.028+0000 7f2b52b5d700 1 Processor -- start 2024-04-25T11:03:03.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.028+0000 7f2b52b5d700 1 -- start start 2024-04-25T11:03:03.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b52b5d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b52b5d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c12e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b52b5d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2b4c12ef30 0x7f2b4c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b52b5d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b4c097f90 con 0x7f2b4c12ef30 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b52b5d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0700e0 con 0x7f2b4c06f200 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b52b5d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c06fe70 con 0x7f2b4c06e8e0 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c12e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c12e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49526/0 (socket says 172.21.15.155:49526) 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c06ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52422/0 (socket says 172.21.15.155:52422) 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4bfff700 1 -- 172.21.15.155:0/4093925237 learned_addr learned my addr 172.21.15.155:0/4093925237 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b510fa700 1 --2- 172.21.15.155:0/4093925237 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c12ef30 0x7f2b4c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 -- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 msgr2=0x7f2b4c12e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 --2- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c12e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 -- 172.21.15.155:0/4093925237 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c12ef30 msgr2=0x7f2b4c12f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 --2- 172.21.15.155:0/4093925237 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c12ef30 0x7f2b4c12f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 -- 172.21.15.155:0/4093925237 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c06c740 con 0x7f2b4c06e8e0 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4bfff700 1 --2- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c12e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b510fa700 1 --2- 172.21.15.155:0/4093925237 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c12ef30 0x7f2b4c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b508f9700 1 --2- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c06ecc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009870 tx=0x7f2b40009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc88141a21a22dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4b7fe700 1 -- 172.21.15.155:0/4093925237 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40011830 con 0x7f2b4c06e8e0 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4b7fe700 1 -- 172.21.15.155:0/4093925237 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b40011990 con 0x7f2b4c06e8e0 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.029+0000 7f2b4b7fe700 1 -- 172.21.15.155:0/4093925237 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40011d20 con 0x7f2b4c06e8e0 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 -- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 msgr2=0x7f2b4c06ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c06ecc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009870 tx=0x7f2b40009b80 comp rx=0 tx=0).stop 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 -- 172.21.15.155:0/4093925237 shutdown_connections 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/4093925237 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c12ef30 0x7f2b4c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c06ecc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/4093925237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c12e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 -- 172.21.15.155:0/4093925237 >> 172.21.15.155:0/4093925237 conn(0x7f2b4c09dc90 msgr2=0x7f2b4c09e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 -- 172.21.15.155:0/4093925237 shutdown_connections 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.030+0000 7f2b52b5d700 1 -- 172.21.15.155:0/4093925237 wait complete. 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 Processor -- start 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 -- start start 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c1c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c1c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c12ef30 0x7f2b4c1c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c08f9c0 con 0x7f2b4c12ef30 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c090b50 con 0x7f2b4c06f200 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b52b5d700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c135820 con 0x7f2b4c06e8e0 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b508f9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c1c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b4bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c1c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b508f9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c1c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52428/0 (socket says 172.21.15.155:52428) 2024-04-25T11:03:03.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b4bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c1c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49534/0 (socket says 172.21.15.155:49534) 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b508f9700 1 -- 172.21.15.155:0/1408915948 learned_addr learned my addr 172.21.15.155:0/1408915948 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.031+0000 7f2b510fa700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c12ef30 0x7f2b4c1c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b4bfff700 1 -- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 msgr2=0x7f2b4c1c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b4bfff700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c1c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b4bfff700 1 -- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c12ef30 msgr2=0x7f2b4c1c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b4bfff700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c12ef30 0x7f2b4c1c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b4bfff700 1 -- 172.21.15.155:0/1408915948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38009550 con 0x7f2b4c06f200 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b508f9700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c1c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b510fa700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c12ef30 0x7f2b4c1c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b4bfff700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c1c35e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c06d5c0 tx=0x7f2b3800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004030 con 0x7f2b4c06f200 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b38004190 con 0x7f2b4c06f200 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b44009a70 con 0x7f2b4c06f200 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004520 con 0x7f2b4c06f200 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.032+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b400094a0 con 0x7f2b4c06f200 2024-04-25T11:03:03.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.033+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f2b380036a0 con 0x7f2b4c06f200 2024-04-25T11:03:03.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.033+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4c062510 con 0x7f2b4c06f200 2024-04-25T11:03:03.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.033+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f2b3805c9a0 con 0x7f2b4c06f200 2024-04-25T11:03:03.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.037+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f2b3803a3e0 con 0x7f2b4c06f200 2024-04-25T11:03:03.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.195+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b4c072780 con 0x7f2b300475f0 2024-04-25T11:03:03.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.197+0000 7f2b497fa700 1 -- 172.21.15.155:0/1408915948 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34599 (unknown 2965378022 0 1829076952) 0x7f2b40011630 con 0x7f2b300475f0 2024-04-25T11:03:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 >> v1:172.21.15.155:6816/49624 conn(0x7f2b300475f0 legacy=0x7f2b30049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 msgr2=0x7f2b4c1c35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c1c35e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c06d5c0 tx=0x7f2b3800ec80 comp rx=0 tx=0).stop 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 shutdown_connections 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c12ef30 0x7f2b4c1c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f2b4c06e8e0 0x7f2b4c1c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 --2- 172.21.15.155:0/1408915948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06f200 0x7f2b4c1c35e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 >> 172.21.15.155:0/1408915948 conn(0x7f2b4c09dc90 msgr2=0x7f2b4c099d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 shutdown_connections 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.199+0000 7f2b52b5d700 1 -- 172.21.15.155:0/1408915948 wait complete. 2024-04-25T11:03:03.199 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:03.210 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-25T11:03:01.992534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110980,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376380,"statfs":{"total":386547056640,"available":386433413120,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":327696,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":65,"num_write":90,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001116"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224548+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:57.224548+0000","last_peered":"2024-04-25T11:02:57.224548+0000","last_clean":"2024-04-25T11:02:57.224548+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:57.224548+0000","last_undegraded":"2024-04-25T11:02:57.224548+0000","last_fullsized":"2024-04-25T11:02:57.224548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:33.129142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226911+0000","last_change":"2024-04-25T11:02:52.039896+0000","last_active":"2024-04-25T11:02:57.226911+0000","last_peered":"2024-04-25T11:02:57.226911+0000","last_clean":"2024-04-25T11:02:57.226911+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:02:57.226911+0000","last_undegraded":"2024-04-25T11:02:57.226911+0000","last_fullsized":"2024-04-25T11:02:57.226911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.972806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224526+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:57.224526+0000","last_peered":"2024-04-25T11:02:57.224526+0000","last_clean":"2024-04-25T11:02:57.224526+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:57.224526+0000","last_undegraded":"2024-04-25T11:02:57.224526+0000","last_fullsized":"2024-04-25T11:02:57.224526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:24.325806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:23.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":157,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226905+0000","last_change":"2024-04-25T11:02:52.039856+0000","last_active":"2024-04-25T11:02:57.226905+0000","last_peered":"2024-04-25T11:02:57.226905+0000","last_clean":"2024-04-25T11:02:57.226905+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:02:57.226905+0000","last_undegraded":"2024-04-25T11:02:57.226905+0000","last_fullsized":"2024-04-25T11:02:57.226905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224458+0000","last_change":"2024-04-25T11:02:54.049124+0000","last_active":"2024-04-25T11:02:57.224458+0000","last_peered":"2024-04-25T11:02:57.224458+0000","last_clean":"2024-04-25T11:02:57.224458+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:02:57.224458+0000","last_undegraded":"2024-04-25T11:02:57.224458+0000","last_fullsized":"2024-04-25T11:02:57.224458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:18.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T11:02:56.752416+0000","last_change":"2024-04-25T11:02:54.048580+0000","last_active":"2024-04-25T11:02:56.752416+0000","last_peered":"2024-04-25T11:02:56.752416+0000","last_clean":"2024-04-25T11:02:56.752416+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:02:56.752416+0000","last_undegraded":"2024-04-25T11:02:56.752416+0000","last_fullsized":"2024-04-25T11:02:56.752416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:35.802506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.268917+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.092289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226923+0000","last_change":"2024-04-25T11:02:52.040177+0000","last_active":"2024-04-25T11:02:57.226923+0000","last_peered":"2024-04-25T11:02:57.226923+0000","last_clean":"2024-04-25T11:02:57.226923+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:02:57.226923+0000","last_undegraded":"2024-04-25T11:02:57.226923+0000","last_fullsized":"2024-04-25T11:02:57.226923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:54.647842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182403,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]}]},{"osd":2,"up_from":24,"seq":103079215108,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":0,"up_from":22,"seq":94489280516,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":23,"seq":98784247812,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:03.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:03:03.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.331+0000 7fa496e87700 1 Processor -- start 2024-04-25T11:03:03.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.331+0000 7fa496e87700 1 -- start start 2024-04-25T11:03:03.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa496e87700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4900974a0 0x7fa490097880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa496e87700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 0x7fa490070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa496e87700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 0x7fa490071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa496e87700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49006c950 con 0x7fa490070fc0 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa496e87700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49006c6e0 con 0x7fa4900974a0 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa496e87700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa490071d50 con 0x7fa49006d0c0 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa494c23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4900974a0 0x7fa490097880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 0x7fa490070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa495424700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 0x7fa490071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa495424700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 0x7fa490071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56600/0 (socket says 172.21.15.155:56600) 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa494c23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4900974a0 0x7fa490097880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49536/0 (socket says 172.21.15.155:49536) 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa495424700 1 -- 172.21.15.155:0/2328545230 learned_addr learned my addr 172.21.15.155:0/2328545230 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 0x7fa490070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52440/0 (socket says 172.21.15.155:52440) 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 -- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4900974a0 msgr2=0x7fa490097880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:03.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 --2- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4900974a0 0x7fa490097880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 -- 172.21.15.155:0/2328545230 >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 msgr2=0x7fa490071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 --2- 172.21.15.155:0/2328545230 >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 0x7fa490071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 -- 172.21.15.155:0/2328545230 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa490072810 con 0x7fa49006d0c0 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa495424700 1 --2- 172.21.15.155:0/2328545230 >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 0x7fa490071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48ffff700 1 --2- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 0x7fa490070a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009840 tx=0x7fa47c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d94f3cec5ddd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48f7fe700 1 -- 172.21.15.155:0/2328545230 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c011830 con 0x7fa49006d0c0 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48f7fe700 1 -- 172.21.15.155:0/2328545230 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c011990 con 0x7fa49006d0c0 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.332+0000 7fa48f7fe700 1 -- 172.21.15.155:0/2328545230 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c011d20 con 0x7fa49006d0c0 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 -- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 msgr2=0x7fa490070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 --2- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 0x7fa490070a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009840 tx=0x7fa47c009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 -- 172.21.15.155:0/2328545230 shutdown_connections 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 --2- 172.21.15.155:0/2328545230 >> v2:172.21.15.155:3300/0 conn(0x7fa490070fc0 0x7fa490071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 --2- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4900974a0 0x7fa490097880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 --2- 172.21.15.155:0/2328545230 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa49006d0c0 0x7fa490070a80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 -- 172.21.15.155:0/2328545230 >> 172.21.15.155:0/2328545230 conn(0x7fa49009dc90 msgr2=0x7fa49009e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:03.336 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 -- 172.21.15.155:0/2328545230 shutdown_connections 2024-04-25T11:03:03.336 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.333+0000 7fa496e87700 1 -- 172.21.15.155:0/2328545230 wait complete. 2024-04-25T11:03:03.336 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 Processor -- start 2024-04-25T11:03:03.336 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 -- start start 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa49006d0c0 0x7fa4901bf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 0x7fa4901bf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4900974a0 0x7fa49008b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49008e120 con 0x7fa4900974a0 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa490128380 con 0x7fa49006d0c0 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa496e87700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa49008cc50 con 0x7fa490070fc0 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 0x7fa4901bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa495424700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4900974a0 0x7fa49008b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa494c23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa49006d0c0 0x7fa4901bf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:03.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 0x7fa4901bf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52446/0 (socket says 172.21.15.155:52446) 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa495424700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4900974a0 0x7fa49008b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56606/0 (socket says 172.21.15.155:56606) 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.334+0000 7fa48ffff700 1 -- 172.21.15.155:0/1058799172 learned_addr learned my addr 172.21.15.155:0/1058799172 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48ffff700 1 -- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa49006d0c0 msgr2=0x7fa4901bf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48ffff700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa49006d0c0 0x7fa4901bf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48ffff700 1 -- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4900974a0 msgr2=0x7fa49008b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48ffff700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4900974a0 0x7fa49008b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48ffff700 1 -- 172.21.15.155:0/1058799172 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa488009b30 con 0x7fa490070fc0 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa494c23700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa49006d0c0 0x7fa4901bf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48ffff700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 0x7fa4901bf650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0090b0 tx=0x7fa47c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004240 con 0x7fa490070fc0 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c01f070 con 0x7fa490070fc0 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c0094a0 con 0x7fa490070fc0 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0036a0 con 0x7fa490070fc0 2024-04-25T11:03:03.339 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.335+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49008b540 con 0x7fa490070fc0 2024-04-25T11:03:03.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.336+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490062510 con 0x7fa490070fc0 2024-04-25T11:03:03.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.336+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa47c021030 con 0x7fa490070fc0 2024-04-25T11:03:03.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.336+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fa47c05d6f0 con 0x7fa490070fc0 2024-04-25T11:03:03.340 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.340+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fa47c025550 con 0x7fa490070fc0 2024-04-25T11:03:03.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.501+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa49008d490 con 0x7fa490070fc0 2024-04-25T11:03:03.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.502+0000 7fa48d7fa700 1 -- 172.21.15.155:0/1058799172 <== mon.2 v2:172.21.15.155:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+8812 (secure 0 0 0) 0x7fa47c025550 con 0x7fa490070fc0 2024-04-25T11:03:03.501 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:03.502 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":29,"fsid":"f023c376-0581-4e8c-aaca-f3f02914116c","created":"2024-04-25T11:00:50.077657+0000","modified":"2024-04-25T11:02:58.221121+0000","last_up_change":"2024-04-25T11:02:53.034844+0000","last_in_change":"2024-04-25T11:00:51.430090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:00:57.443026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T11:00:59.312325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1a66333-2acf-451b-b7d0-2dfa2e55600f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":16,"up_from":22,"up_thru":22,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":51324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":51324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":51324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":51324}]},"public_addr":"172.21.15.155:6800/51324","cluster_addr":"172.21.15.155:6801/51324","heartbeat_back_addr":"172.21.15.155:6803/51324","heartbeat_front_addr":"172.21.15.155:6802/51324","state":["exists","up"]},{"osd":1,"uuid":"7051096c-4e97-4ace-9814-455da6c8a06b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":23,"up_thru":23,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":51418}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":51418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":51418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":51418}]},"public_addr":"172.21.15.155:6808/51418","cluster_addr":"172.21.15.155:6809/51418","heartbeat_back_addr":"172.21.15.155:6811/51418","heartbeat_front_addr":"172.21.15.155:6810/51418","state":["exists","up"]},{"osd":2,"uuid":"faf1e6c0-e94a-4bf3-830d-67564885faaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":17,"up_from":24,"up_thru":25,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":51529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":51529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":51529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":51529}]},"public_addr":"172.21.15.155:6804/51529","cluster_addr":"172.21.15.155:6805/51529","heartbeat_back_addr":"172.21.15.155:6807/51529","heartbeat_front_addr":"172.21.15.155:6806/51529","state":["exists","up"]},{"osd":3,"uuid":"4e269fdb-6854-450b-9956-6ab00516289e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":51677}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":51677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":51677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":51677}]},"public_addr":"172.21.15.155:6812/51677","cluster_addr":"172.21.15.155:6813/51677","heartbeat_back_addr":"172.21.15.155:6815/51677","heartbeat_front_addr":"172.21.15.155:6814/51677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.389357+0000","dead_epoch":17},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.579751+0000","dead_epoch":18},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.625639+0000","dead_epoch":19},{"osd":3,"down_stamp":"2024-04-25T11:02:47.002020+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:00:56.361390+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:0/3077341019":"2024-04-26T11:02:39.971273+0000","172.21.15.155:0/966131479":"2024-04-26T11:02:39.971273+0000","172.21.15.155:6816/43298":"2024-04-26T11:02:39.971273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:03:03.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.503+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 >> v1:172.21.15.155:6816/49624 conn(0x7fa4740475f0 legacy=0x7fa474049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.503+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 msgr2=0x7fa4901bf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.503+0000 7fa496e87700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 0x7fa4901bf650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0090b0 tx=0x7fa47c004750 comp rx=0 tx=0).stop 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 shutdown_connections 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4900974a0 0x7fa49008b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa49006d0c0 0x7fa4901bf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 --2- 172.21.15.155:0/1058799172 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa490070fc0 0x7fa4901bf650 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 >> 172.21.15.155:0/1058799172 conn(0x7fa49009dc90 msgr2=0x7fa49006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 shutdown_connections 2024-04-25T11:03:03.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:03.504+0000 7fa496e87700 1 -- 172.21.15.155:0/1058799172 wait complete. 2024-04-25T11:03:04.516 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T11:03:04.516 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:03:04.652 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 Processor -- start 2024-04-25T11:03:04.652 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 -- start start 2024-04-25T11:03:04.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc129500 0x7fe3dc1298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc06ab70 0x7fe3dc12c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc072ee0 con 0x7fe3dc06d0a0 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06cb80 con 0x7fe3dc129500 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3e0fd3700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc129e20 con 0x7fe3dc06ab70 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3dad9d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe3dc06d0a0 0x7fe3dc12c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53504/0 (socket says 172.21.15.155:53504) 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3d9d9b700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc06ab70 0x7fe3dc12c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3da59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc129500 0x7fe3dc1298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.653+0000 7fe3dad9d700 1 -- 172.21.15.155:0/4156936962 learned_addr learned my addr 172.21.15.155:0/4156936962 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3d959a700 1 -- 172.21.15.155:0/4156936962 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113406069 0 0) 0x7fe3dc072ee0 con 0x7fe3dc06d0a0 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3d959a700 1 -- 172.21.15.155:0/4156936962 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c40035b0 con 0x7fe3dc06d0a0 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3da59c700 1 -- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc06ab70 msgr2=0x7fe3dc12c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3da59c700 1 --2- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc06ab70 0x7fe3dc12c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3da59c700 1 -- 172.21.15.155:0/4156936962 >> v1:172.21.15.155:6789/0 conn(0x7fe3dc06d0a0 legacy=0x7fe3dc12c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3da59c700 1 -- 172.21.15.155:0/4156936962 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc129f90 con 0x7fe3dc129500 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3d9d9b700 1 --2- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc06ab70 0x7fe3dc12c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3da59c700 1 --2- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc129500 0x7fe3dc1298e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0098a0 tx=0x7fe3cc009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35311d5282346b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3d959a700 1 -- 172.21.15.155:0/4156936962 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc011830 con 0x7fe3dc129500 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3d959a700 1 -- 172.21.15.155:0/4156936962 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc011990 con 0x7fe3dc129500 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.654+0000 7fe3d959a700 1 -- 172.21.15.155:0/4156936962 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc011d20 con 0x7fe3dc129500 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc129500 msgr2=0x7fe3dc1298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 --2- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc129500 0x7fe3dc1298e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0098a0 tx=0x7fe3cc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4156936962 shutdown_connections 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 --2- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc129500 0x7fe3dc1298e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 --2- 172.21.15.155:0/4156936962 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc06ab70 0x7fe3dc12c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4156936962 >> 172.21.15.155:0/4156936962 conn(0x7fe3dc09a140 msgr2=0x7fe3dc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4156936962 shutdown_connections 2024-04-25T11:03:04.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.655+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4156936962 wait complete. 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 Processor -- start 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 -- start start 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3dc06ab70 0x7fe3dc06dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc06d0a0 0x7fe3dc06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc129500 0x7fe3dc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06f6b0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc072560 con 0x7fe3dc06d0a0 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3e0fd3700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06fe20 con 0x7fe3dc129500 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3dc06ab70 0x7fe3dc06dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3dad9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc129500 0x7fe3dc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3d9d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc06d0a0 0x7fe3dc06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3dc06ab70 0x7fe3dc06dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56614/0 (socket says 172.21.15.155:56614) 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3d9d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc06d0a0 0x7fe3dc06e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49568/0 (socket says 172.21.15.155:49568) 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3dad9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc129500 0x7fe3dc06e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52462/0 (socket says 172.21.15.155:52462) 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 -- 172.21.15.155:0/4002840914 learned_addr learned my addr 172.21.15.155:0/4002840914 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 -- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc129500 msgr2=0x7fe3dc06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 --2- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fe3dc129500 0x7fe3dc06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 -- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc06d0a0 msgr2=0x7fe3dc06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.656+0000 7fe3da59c700 1 --2- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc06d0a0 0x7fe3dc06e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3da59c700 1 -- 172.21.15.155:0/4002840914 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00094a0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3d9d9b700 1 --2- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3dc06d0a0 0x7fe3dc06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3da59c700 1 --2- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3dc06ab70 0x7fe3dc06dbf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0090b0 tx=0x7fe3cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc022070 con 0x7fe3dc06ab70 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc004250 con 0x7fe3dc06ab70 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4002840914 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc0094a0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0045c0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.657+0000 7fe3e0fd3700 1 -- 172.21.15.155:0/4002840914 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc128d40 con 0x7fe3dc06ab70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.658+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3b4000fb0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.658+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fe3cc01d7c0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.658+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fe3cc061860 con 0x7fe3dc06ab70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.658+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 --> v1:172.21.15.155:6800/51324 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3bc051a40 con 0x7fe3bc04dd70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.658+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3cc0662c0 con 0x7fe3dc06ab70 2024-04-25T11:03:04.661 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.659+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== osd.0 v1:172.21.15.155:6800/51324 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7fe3bc051a40 con 0x7fe3bc04dd70 2024-04-25T11:03:04.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.673+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 --> v1:172.21.15.155:6800/51324 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe3b4002d50 con 0x7fe3bc04dd70 2024-04-25T11:03:04.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.673+0000 7fe3caffd700 1 -- 172.21.15.155:0/4002840914 <== osd.0 v1:172.21.15.155:6800/51324 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7fe3b4002d50 con 0x7fe3bc04dd70 2024-04-25T11:03:04.673 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-25T11:03:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:03:04.673 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-25T11:03:04.674 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 >> v1:172.21.15.155:6800/51324 conn(0x7fe3bc04dd70 legacy=0x7fe3bc0501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 >> v1:172.21.15.155:6816/49624 conn(0x7fe3bc0475f0 legacy=0x7fe3bc049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3dc06ab70 msgr2=0x7fe3dc06dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 --2- 172.21.15.155:0/4002840914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3dc06ab70 0x7fe3dc06dbf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0090b0 tx=0x7fe3cc004750 comp rx=0 tx=0).stop 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3dad9d700 1 -- 172.21.15.155:0/4002840914 reap_dead start 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 shutdown_connections 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 >> 172.21.15.155:0/4002840914 conn(0x7fe3dc09a140 msgr2=0x7fe3dc1240e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 shutdown_connections 2024-04-25T11:03:04.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.674+0000 7fe3c8ff9700 1 -- 172.21.15.155:0/4002840914 wait complete. 2024-04-25T11:03:04.685 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T11:03:04.808 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.806+0000 7f0d4f863700 1 Processor -- start 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 -- start start 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 0x7f0d481286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 0x7f0d4812b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d480728f0 con 0x7f0d481282e0 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4806f960 con 0x7f0d4812b220 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4f863700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48128c00 con 0x7f0d4812bb80 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4d5ff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 0x7f0d481286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4de00700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 0x7f0d4812b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4d5ff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 0x7f0d481286c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56618/0 (socket says 172.21.15.155:56618) 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4de00700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481325b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52466/0 (socket says 172.21.15.155:52466) 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 0x7f0d4812b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49584/0 (socket says 172.21.15.155:49584) 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4d5ff700 1 -- 172.21.15.155:0/3031349268 learned_addr learned my addr 172.21.15.155:0/3031349268 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:04.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 -- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 msgr2=0x7f0d481325b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 --2- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481325b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 -- 172.21.15.155:0/3031349268 >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 msgr2=0x7f0d481286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 --2- 172.21.15.155:0/3031349268 >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 0x7f0d481286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4cdfe700 1 -- 172.21.15.155:0/3031349268 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4806fe90 con 0x7f0d4812b220 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.807+0000 7f0d4d5ff700 1 --2- 172.21.15.155:0/3031349268 >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 0x7f0d481286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4cdfe700 1 --2- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 0x7f0d4812b640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009840 tx=0x7f0d44009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0463941639cda08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d3ffff700 1 -- 172.21.15.155:0/3031349268 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44011830 con 0x7f0d4812b220 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d3ffff700 1 -- 172.21.15.155:0/3031349268 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44011990 con 0x7f0d4812b220 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d3ffff700 1 -- 172.21.15.155:0/3031349268 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44011d20 con 0x7f0d4812b220 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4f863700 1 -- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 msgr2=0x7f0d4812b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4f863700 1 --2- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 0x7f0d4812b640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009840 tx=0x7f0d44009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4f863700 1 -- 172.21.15.155:0/3031349268 shutdown_connections 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4f863700 1 --2- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4f863700 1 --2- 172.21.15.155:0/3031349268 >> v2:172.21.15.155:3300/0 conn(0x7f0d481282e0 0x7f0d481286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.808+0000 7f0d4f863700 1 --2- 172.21.15.155:0/3031349268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4812b220 0x7f0d4812b640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.809+0000 7f0d4f863700 1 -- 172.21.15.155:0/3031349268 >> 172.21.15.155:0/3031349268 conn(0x7f0d4809dac0 msgr2=0x7f0d4809ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:04.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.809+0000 7f0d4f863700 1 -- 172.21.15.155:0/3031349268 shutdown_connections 2024-04-25T11:03:04.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.809+0000 7f0d4f863700 1 -- 172.21.15.155:0/3031349268 wait complete. 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.809+0000 7f0d4f863700 1 Processor -- start 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.809+0000 7f0d4f863700 1 -- start start 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4f863700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 0x7f0d48124650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4f863700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4812b220 0x7f0d48124b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4f863700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481250d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4f863700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48127080 con 0x7f0d4812b220 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4f863700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48126760 con 0x7f0d481282e0 2024-04-25T11:03:04.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4f863700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d481263d0 con 0x7f0d4812bb80 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 0x7f0d48124650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4de00700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 0x7f0d48124650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49600/0 (socket says 172.21.15.155:49600) 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4cdfe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4812b220 0x7f0d48124b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 -- 172.21.15.155:0/27256802 learned_addr learned my addr 172.21.15.155:0/27256802 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 -- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 msgr2=0x7f0d481250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481250d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 -- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4812b220 msgr2=0x7f0d48124b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4812b220 0x7f0d48124b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 -- 172.21.15.155:0/27256802 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d38009550 con 0x7f0d481282e0 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4cdfe700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4812b220 0x7f0d48124b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4de00700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d4d5ff700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 0x7f0d48124650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800e9b0 tx=0x7f0d3800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:04.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800ee50 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.810+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3800c850 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.811+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d440094a0 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.811+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800cbc0 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.811+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d48127250 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.812+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d48099800 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.812+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0d3800cd20 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.812+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f0d3805c830 con 0x7f0d481282e0 2024-04-25T11:03:04.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.815+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f0d3803a4e0 con 0x7f0d481282e0 2024-04-25T11:03:04.976 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.976+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d48070d90 con 0x7f0d2c0475f0 2024-04-25T11:03:04.976 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.976+0000 7f0d3dffb700 1 -- 172.21.15.155:0/27256802 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7f0d44011630 con 0x7f0d2c0475f0 2024-04-25T11:03:04.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 >> v1:172.21.15.155:6816/49624 conn(0x7f0d2c0475f0 legacy=0x7f0d2c049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 msgr2=0x7f0d48124650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:04.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 0x7f0d48124650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800e9b0 tx=0x7f0d3800ecc0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 shutdown_connections 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d4812bb80 0x7f0d481250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d481282e0 0x7f0d48124650 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 --2- 172.21.15.155:0/27256802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4812b220 0x7f0d48124b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 >> 172.21.15.155:0/27256802 conn(0x7f0d4809dac0 msgr2=0x7f0d480977c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 shutdown_connections 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:04.978+0000 7f0d4f863700 1 -- 172.21.15.155:0/27256802 wait complete. 2024-04-25T11:03:04.979 INFO:teuthology.orchestra.run.smithi155.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T11:03:04.989 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T11:03:04.989 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.115+0000 7f4d9e038700 1 Processor -- start 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.115+0000 7f4d9e038700 1 -- start start 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.115+0000 7f4d9e038700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806e910 0x7f4d9806ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.115+0000 7f4d9e038700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 0x7f4d9812e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d9e038700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9806c970 con 0x7f4d9812ef30 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d9e038700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9806d300 con 0x7f4d9806e910 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d9e038700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9806dcf0 con 0x7f4d9806f230 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 0x7f4d9812e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d977fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806e910 0x7f4d9806ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 0x7f4d9812e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52494/0 (socket says 172.21.15.155:52494) 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d96ffd700 1 -- 172.21.15.155:0/1894058980 learned_addr learned my addr 172.21.15.155:0/1894058980 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d97fff700 1 --1- 172.21.15.155:0/1894058980 >> v1:172.21.15.155:6789/0 conn(0x7f4d9812ef30 0x7f4d98135c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53508/0 (socket says 172.21.15.155:53508) 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d977fe700 1 -- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 msgr2=0x7f4d9812e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d977fe700 1 --2- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 0x7f4d9812e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d977fe700 1 -- 172.21.15.155:0/1894058980 >> v1:172.21.15.155:6789/0 conn(0x7f4d9812ef30 legacy=0x7f4d98135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d967fc700 1 -- 172.21.15.155:0/1894058980 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456764676 0 0) 0x7f4d9806c970 con 0x7f4d9812ef30 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d977fe700 1 -- 172.21.15.155:0/1894058980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9806c6e0 con 0x7f4d9806e910 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d96ffd700 1 --2- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 0x7f4d9812e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d977fe700 1 --2- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806e910 0x7f4d9806ecf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009870 tx=0x7f4d90009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94feb346e75f8be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d967fc700 1 -- 172.21.15.155:0/1894058980 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90011830 con 0x7f4d9806e910 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d967fc700 1 -- 172.21.15.155:0/1894058980 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90011990 con 0x7f4d9806e910 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.116+0000 7f4d967fc700 1 -- 172.21.15.155:0/1894058980 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90011d20 con 0x7f4d9806e910 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 -- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806e910 msgr2=0x7f4d9806ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.118 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 --2- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806e910 0x7f4d9806ecf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009870 tx=0x7f4d90009b80 comp rx=0 tx=0).stop 2024-04-25T11:03:05.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 -- 172.21.15.155:0/1894058980 shutdown_connections 2024-04-25T11:03:05.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 --2- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806e910 0x7f4d9806ecf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 --2- 172.21.15.155:0/1894058980 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9806f230 0x7f4d9812e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 -- 172.21.15.155:0/1894058980 >> 172.21.15.155:0/1894058980 conn(0x7f4d9809dc90 msgr2=0x7f4d9809e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 -- 172.21.15.155:0/1894058980 shutdown_connections 2024-04-25T11:03:05.119 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.117+0000 7f4d9e038700 1 -- 172.21.15.155:0/1894058980 wait complete. 2024-04-25T11:03:05.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 Processor -- start 2024-04-25T11:03:05.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 -- start start 2024-04-25T11:03:05.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9806e910 0x7f4d980945d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806f230 0x7f4d98094b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9812ef30 0x7f4d98095050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.121 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981230e0 con 0x7f4d9806e910 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98122d10 con 0x7f4d9806f230 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d9e038700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98096800 con 0x7f4d9812ef30 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d97fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9812ef30 0x7f4d98095050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9806e910 0x7f4d980945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806f230 0x7f4d98094b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9806e910 0x7f4d980945d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56634/0 (socket says 172.21.15.155:56634) 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d97fff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9812ef30 0x7f4d98095050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52504/0 (socket says 172.21.15.155:52504) 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806f230 0x7f4d98094b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49612/0 (socket says 172.21.15.155:49612) 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.118+0000 7f4d977fe700 1 -- 172.21.15.155:0/1229391507 learned_addr learned my addr 172.21.15.155:0/1229391507 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d97fff700 1 -- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806f230 msgr2=0x7f4d98094b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d97fff700 1 --2- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806f230 0x7f4d98094b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d97fff700 1 -- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9806e910 msgr2=0x7f4d980945d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d97fff700 1 --2- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9806e910 0x7f4d980945d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d97fff700 1 -- 172.21.15.155:0/1229391507 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80009550 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d96ffd700 1 --2- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d9806f230 0x7f4d98094b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d97fff700 1 --2- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9812ef30 0x7f4d98095050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800cb30 tx=0x7f4d8800ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800a980 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800ab00 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880070d0 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d900094a0 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.119+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d981c2e30 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.120+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d9806cf80 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.120+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f4d88007230 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.120+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f4d8805c250 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.120+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 --> v1:172.21.15.155:6808/51418 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d78051a40 con 0x7f4d7804dd70 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.120+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d8805e910 con 0x7f4d9812ef30 2024-04-25T11:03:05.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.121+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== osd.1 v1:172.21.15.155:6808/51418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7f4d78051a40 con 0x7f4d7804dd70 2024-04-25T11:03:05.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.135+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 --> v1:172.21.15.155:6808/51418 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4d98071aa0 con 0x7f4d7804dd70 2024-04-25T11:03:05.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.135+0000 7f4d8ffff700 1 -- 172.21.15.155:0/1229391507 <== osd.1 v1:172.21.15.155:6808/51418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f4d98071aa0 con 0x7f4d7804dd70 2024-04-25T11:03:05.135 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-25T11:03:05.135 INFO:teuthology.orchestra.run.smithi155.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:03:05.135 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 >> v1:172.21.15.155:6808/51418 conn(0x7f4d7804dd70 legacy=0x7f4d780501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 >> v1:172.21.15.155:6816/49624 conn(0x7f4d780475f0 legacy=0x7f4d78049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9812ef30 msgr2=0x7f4d98095050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 --2- 172.21.15.155:0/1229391507 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f4d9812ef30 0x7f4d98095050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800cb30 tx=0x7f4d8800ce40 comp rx=0 tx=0).stop 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d97fff700 1 -- 172.21.15.155:0/1229391507 reap_dead start 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 shutdown_connections 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 >> 172.21.15.155:0/1229391507 conn(0x7f4d9809dc90 msgr2=0x7f4d98071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 shutdown_connections 2024-04-25T11:03:05.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.136+0000 7f4d9e038700 1 -- 172.21.15.155:0/1229391507 wait complete. 2024-04-25T11:03:05.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T11:03:05.269 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 Processor -- start 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 -- start start 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 0x7f320c12ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 0x7f320c136930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320c06ab70 con 0x7f320c12f110 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c06d260 con 0x7f320c072750 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3210c08700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f320c067a80 con 0x7f320c12e7f0 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f320a59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 0x7f320c12ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f320ad9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 0x7f320c136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f320a59c700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 0x7f320c12ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52508/0 (socket says 172.21.15.155:52508) 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f3209d9b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f320c12f110 0x7f320c072040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53512/0 (socket says 172.21.15.155:53512) 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f320ad9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 0x7f320c136930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56638/0 (socket says 172.21.15.155:56638) 2024-04-25T11:03:05.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.268+0000 7f320a59c700 1 -- 172.21.15.155:0/4021879764 learned_addr learned my addr 172.21.15.155:0/4021879764 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320a59c700 1 -- 172.21.15.155:0/4021879764 >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 msgr2=0x7f320c136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320a59c700 1 --2- 172.21.15.155:0/4021879764 >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 0x7f320c136930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320a59c700 1 -- 172.21.15.155:0/4021879764 >> v1:172.21.15.155:6789/0 conn(0x7f320c12f110 legacy=0x7f320c072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320a59c700 1 -- 172.21.15.155:0/4021879764 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c12e5f0 con 0x7f320c12e7f0 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320ad9d700 1 --2- 172.21.15.155:0/4021879764 >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 0x7f320c136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320a59c700 1 --2- 172.21.15.155:0/4021879764 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 0x7f320c12ebd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0098f0 tx=0x7f31fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d834ed0594b3259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320959a700 1 -- 172.21.15.155:0/4021879764 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc011830 con 0x7f320c12e7f0 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320959a700 1 -- 172.21.15.155:0/4021879764 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc011990 con 0x7f320c12e7f0 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f320959a700 1 -- 172.21.15.155:0/4021879764 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc011d20 con 0x7f320c12e7f0 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f3210c08700 1 -- 172.21.15.155:0/4021879764 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 msgr2=0x7f320c12ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.269+0000 7f3210c08700 1 --2- 172.21.15.155:0/4021879764 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 0x7f320c12ebd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0098f0 tx=0x7f31fc009c00 comp rx=0 tx=0).stop 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 -- 172.21.15.155:0/4021879764 shutdown_connections 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 --2- 172.21.15.155:0/4021879764 >> v2:172.21.15.155:3300/0 conn(0x7f320c072750 0x7f320c136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 --2- 172.21.15.155:0/4021879764 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12e7f0 0x7f320c12ebd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 -- 172.21.15.155:0/4021879764 >> 172.21.15.155:0/4021879764 conn(0x7f320c09a220 msgr2=0x7f320c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 -- 172.21.15.155:0/4021879764 shutdown_connections 2024-04-25T11:03:05.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 -- 172.21.15.155:0/4021879764 wait complete. 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.270+0000 7f3210c08700 1 Processor -- start 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 -- start start 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 0x7f320c1c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f320c12e7f0 0x7f320c1c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12f110 0x7f320c1c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c125b40 con 0x7f320c072750 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c125700 con 0x7f320c12e7f0 2024-04-25T11:03:05.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3210c08700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1247d0 con 0x7f320c12f110 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 0x7f320c1c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320ad9d700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12f110 0x7f320c1c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 0x7f320c1c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56654/0 (socket says 172.21.15.155:56654) 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 -- 172.21.15.155:0/849745444 learned_addr learned my addr 172.21.15.155:0/849745444 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3209d9b700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f320c12e7f0 0x7f320c1c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 -- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12f110 msgr2=0x7f320c1c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12f110 0x7f320c1c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 -- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f320c12e7f0 msgr2=0x7f320c1c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f320c12e7f0 0x7f320c1c3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 -- 172.21.15.155:0/849745444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204009a70 con 0x7f320c072750 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f3209d9b700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f320c12e7f0 0x7f320c1c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320ad9d700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12f110 0x7f320c1c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.271+0000 7f320a59c700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 0x7f320c1c30e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000c00 tx=0x7f31fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.272+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0043c0 con 0x7f320c072750 2024-04-25T11:03:05.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.272+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc01f070 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.272+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc009550 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.272+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0038c0 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.272+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c124fd0 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.273+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c123e80 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.273+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f31fc021030 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.273+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f31fc05db10 con 0x7f320c072750 2024-04-25T11:03:05.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.277+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f320c123e80 con 0x7f320c072750 2024-04-25T11:03:05.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.437+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f320c092260 con 0x7f31ec0475f0 2024-04-25T11:03:05.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.437+0000 7f31faffd700 1 -- 172.21.15.155:0/849745444 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7f32000016e0 con 0x7f31ec0475f0 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 >> v1:172.21.15.155:6816/49624 conn(0x7f31ec0475f0 legacy=0x7f31ec049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 msgr2=0x7f320c1c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 0x7f320c1c30e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000c00 tx=0x7f31fc004750 comp rx=0 tx=0).stop 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 shutdown_connections 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f320c12f110 0x7f320c1c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f320c072750 0x7f320c1c30e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 --2- 172.21.15.155:0/849745444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f320c12e7f0 0x7f320c1c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 >> 172.21.15.155:0/849745444 conn(0x7f320c09a220 msgr2=0x7f320c0945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 shutdown_connections 2024-04-25T11:03:05.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.439+0000 7f3210c08700 1 -- 172.21.15.155:0/849745444 wait complete. 2024-04-25T11:03:05.440 INFO:teuthology.orchestra.run.smithi155.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T11:03:05.450 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T11:03:05.451 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:03:05.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 Processor -- start 2024-04-25T11:03:05.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 -- start start 2024-04-25T11:03:05.576 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5d8c070b00 0x7f5d8c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c134da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c071ec0 con 0x7f5d8c071420 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c06dd80 con 0x7f5d8c070b00 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d92edb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c06d6c0 con 0x7f5d8c12d250 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d90c77700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5d8c070b00 0x7f5d8c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d91478700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d8bfff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5d8c071420 0x7f5d8c12cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53514/0 (socket says 172.21.15.155:53514) 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.575+0000 7f5d8bfff700 1 -- 172.21.15.155:0/2196965285 learned_addr learned my addr 172.21.15.155:0/2196965285 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d90c77700 1 -- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 msgr2=0x7f5d8c134da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d90c77700 1 --2- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c134da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d90c77700 1 -- 172.21.15.155:0/2196965285 >> v1:172.21.15.155:6789/0 conn(0x7f5d8c071420 legacy=0x7f5d8c12cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d90c77700 1 -- 172.21.15.155:0/2196965285 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c06d3b0 con 0x7f5d8c070b00 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d8b7fe700 1 -- 172.21.15.155:0/2196965285 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797843580 0 0) 0x7f5d78001110 con 0x7f5d8c071420 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d91478700 1 --2- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d90c77700 1 --2- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c070ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009840 tx=0x7f5d80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e634e2699b292d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d8b7fe700 1 -- 172.21.15.155:0/2196965285 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80011830 con 0x7f5d8c070b00 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d8b7fe700 1 -- 172.21.15.155:0/2196965285 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80011990 con 0x7f5d8c070b00 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d8b7fe700 1 -- 172.21.15.155:0/2196965285 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80011d20 con 0x7f5d8c070b00 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 -- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 msgr2=0x7f5d8c070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 --2- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c070ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009840 tx=0x7f5d80009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 -- 172.21.15.155:0/2196965285 shutdown_connections 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 --2- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c070ee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 --2- 172.21.15.155:0/2196965285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 -- 172.21.15.155:0/2196965285 >> 172.21.15.155:0/2196965285 conn(0x7f5d8c09de00 msgr2=0x7f5d8c09e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 -- 172.21.15.155:0/2196965285 shutdown_connections 2024-04-25T11:03:05.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.576+0000 7f5d92edb700 1 -- 172.21.15.155:0/2196965285 wait complete. 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.577+0000 7f5d92edb700 1 Processor -- start 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.577+0000 7f5d92edb700 1 -- start start 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d92edb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d92edb700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5d8c071420 0x7f5d8c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d92edb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d92edb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0960b0 con 0x7f5d8c070b00 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d92edb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c129670 con 0x7f5d8c12d250 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d92edb700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0702a0 con 0x7f5d8c071420 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d91478700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5d8c071420 0x7f5d8c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56672/0 (socket says 172.21.15.155:56672) 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d91478700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c06f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49636/0 (socket says 172.21.15.155:49636) 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 -- 172.21.15.155:0/3252051613 learned_addr learned my addr 172.21.15.155:0/3252051613 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:05.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 -- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5d8c071420 msgr2=0x7f5d8c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 --2- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f5d8c071420 0x7f5d8c06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 -- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 msgr2=0x7f5d8c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 --2- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c06f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 -- 172.21.15.155:0/3252051613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84009a70 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d91478700 1 --2- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d8c12d250 0x7f5d8c06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.578+0000 7f5d90c77700 1 --2- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c06e8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009fd0 tx=0x7f5d80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.579+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80004550 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.579+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8001e070 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.579+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d800094a0 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.579+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d800038e0 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.579+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c095d40 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.579+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d8c06c6e0 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.580+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5d80003a40 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.580+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f5d8005d5f0 con 0x7f5d8c070b00 2024-04-25T11:03:05.582 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.580+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 --> v1:172.21.15.155:6804/51529 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d70051a70 con 0x7f5d7004dda0 2024-04-25T11:03:05.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.580+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8005fcb0 con 0x7f5d8c070b00 2024-04-25T11:03:05.583 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.581+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== osd.2 v1:172.21.15.155:6804/51529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7f5d70051a70 con 0x7f5d7004dda0 2024-04-25T11:03:05.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 --> v1:172.21.15.155:6804/51529 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5d8c071860 con 0x7f5d7004dda0 2024-04-25T11:03:05.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d897fa700 1 -- 172.21.15.155:0/3252051613 <== osd.2 v1:172.21.15.155:6804/51529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f5d8400f040 con 0x7f5d7004dda0 2024-04-25T11:03:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-25T11:03:05.595 INFO:teuthology.orchestra.run.smithi155.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:03:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-25T11:03:05.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 >> v1:172.21.15.155:6804/51529 conn(0x7f5d7004dda0 legacy=0x7f5d70050200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 >> v1:172.21.15.155:6816/49624 conn(0x7f5d70047620 legacy=0x7f5d70049ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 msgr2=0x7f5d8c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 --2- 172.21.15.155:0/3252051613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d8c070b00 0x7f5d8c06e8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009fd0 tx=0x7f5d80004750 comp rx=0 tx=0).stop 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d91478700 1 -- 172.21.15.155:0/3252051613 reap_dead start 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 shutdown_connections 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 >> 172.21.15.155:0/3252051613 conn(0x7f5d8c09de00 msgr2=0x7f5d8c0983e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 shutdown_connections 2024-04-25T11:03:05.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.595+0000 7f5d92edb700 1 -- 172.21.15.155:0/3252051613 wait complete. 2024-04-25T11:03:05.606 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T11:03:05.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 Processor -- start 2024-04-25T11:03:05.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 -- start start 2024-04-25T11:03:05.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b80960a0 0x7f29b8095090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f29b8134220 0x7f29b8136610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80969b0 con 0x7f29b806a900 2024-04-25T11:03:05.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b8066870 con 0x7f29b8134220 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.727+0000 7f29bf4b7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b809ce50 con 0x7f29b80960a0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bda54700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f29b8134220 0x7f29b8136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bca52700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b80960a0 0x7f29b8095090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bd253700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f29b806a900 0x7f29b8094980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53524/0 (socket says 172.21.15.155:53524) 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bd253700 1 -- 172.21.15.155:0/4216840714 learned_addr learned my addr 172.21.15.155:0/4216840714 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bca52700 1 -- 172.21.15.155:0/4216840714 >> v2:172.21.15.155:3300/0 conn(0x7f29b8134220 msgr2=0x7f29b8136610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bca52700 1 --2- 172.21.15.155:0/4216840714 >> v2:172.21.15.155:3300/0 conn(0x7f29b8134220 0x7f29b8136610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bca52700 1 -- 172.21.15.155:0/4216840714 >> v1:172.21.15.155:6789/0 conn(0x7f29b806a900 legacy=0x7f29b8094980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bca52700 1 -- 172.21.15.155:0/4216840714 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80955d0 con 0x7f29b80960a0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bda54700 1 --2- 172.21.15.155:0/4216840714 >> v2:172.21.15.155:3300/0 conn(0x7f29b8134220 0x7f29b8136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29bca52700 1 --2- 172.21.15.155:0/4216840714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b80960a0 0x7f29b8095090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009840 tx=0x7f29b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f6a3226cc93e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29affff700 1 -- 172.21.15.155:0/4216840714 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4011830 con 0x7f29b80960a0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29affff700 1 -- 172.21.15.155:0/4216840714 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4011990 con 0x7f29b80960a0 2024-04-25T11:03:05.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.728+0000 7f29affff700 1 -- 172.21.15.155:0/4216840714 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4011d20 con 0x7f29b80960a0 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 -- 172.21.15.155:0/4216840714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b80960a0 msgr2=0x7f29b8095090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/4216840714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b80960a0 0x7f29b8095090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009840 tx=0x7f29b4009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 -- 172.21.15.155:0/4216840714 shutdown_connections 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/4216840714 >> v2:172.21.15.155:3300/0 conn(0x7f29b8134220 0x7f29b8136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/4216840714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b80960a0 0x7f29b8095090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 -- 172.21.15.155:0/4216840714 >> 172.21.15.155:0/4216840714 conn(0x7f29b809d480 msgr2=0x7f29b809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 -- 172.21.15.155:0/4216840714 shutdown_connections 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.729+0000 7f29bf4b7700 1 -- 172.21.15.155:0/4216840714 wait complete. 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 Processor -- start 2024-04-25T11:03:05.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 -- start start 2024-04-25T11:03:05.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 0x7f29b8071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f29b80960a0 0x7f29b8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b8134220 0x7f29b8072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:05.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8071350 con 0x7f29b806a900 2024-04-25T11:03:05.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8070080 con 0x7f29b8134220 2024-04-25T11:03:05.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bf4b7700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8070350 con 0x7f29b80960a0 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bd253700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 0x7f29b8071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bda54700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b8134220 0x7f29b8072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bca52700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f29b80960a0 0x7f29b8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bd253700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 0x7f29b8071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56678/0 (socket says 172.21.15.155:56678) 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bd253700 1 -- 172.21.15.155:0/649101392 learned_addr learned my addr 172.21.15.155:0/649101392 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bda54700 1 -- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f29b80960a0 msgr2=0x7f29b8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bda54700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f29b80960a0 0x7f29b8071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bda54700 1 -- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 msgr2=0x7f29b8071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bda54700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 0x7f29b8071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bda54700 1 -- 172.21.15.155:0/649101392 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0009a70 con 0x7f29b8134220 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bca52700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f29b80960a0 0x7f29b8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.730+0000 7f29bd253700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 0x7f29b8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.731+0000 7f29bda54700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b8134220 0x7f29b8072090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29b000ef90 tx=0x7f29b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:05.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.731+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000ca90 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.731+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b000f040 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.731+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b40094a0 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.731+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000ce30 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.731+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b806f720 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.732+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f29b0014480 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.732+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b8062510 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.732+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f29b005cb90 con 0x7f29b8134220 2024-04-25T11:03:05.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.736+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f29b003a610 con 0x7f29b8134220 2024-04-25T11:03:05.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.896+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f29b806f720 con 0x7f299c0475f0 2024-04-25T11:03:05.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.897+0000 7f29adffb700 1 -- 172.21.15.155:0/649101392 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7f29b806f720 con 0x7f299c0475f0 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 >> v1:172.21.15.155:6816/49624 conn(0x7f299c0475f0 legacy=0x7f299c049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b8134220 msgr2=0x7f29b8072090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b8134220 0x7f29b8072090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29b000ef90 tx=0x7f29b000c3a0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 shutdown_connections 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29b8134220 0x7f29b8072090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b806a900 0x7f29b8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 --2- 172.21.15.155:0/649101392 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f29b80960a0 0x7f29b8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:05.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 >> 172.21.15.155:0/649101392 conn(0x7f29b809d480 msgr2=0x7f29b8097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:05.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 shutdown_connections 2024-04-25T11:03:05.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:05.899+0000 7f29bf4b7700 1 -- 172.21.15.155:0/649101392 wait complete. 2024-04-25T11:03:05.900 INFO:teuthology.orchestra.run.smithi155.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T11:03:05.910 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T11:03:05.910 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:03:06.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.034+0000 7fa6b3fea700 1 Processor -- start 2024-04-25T11:03:06.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 -- start start 2024-04-25T11:03:06.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 0x7fa6ac071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 0x7fa6ac12b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 0x7fa6ac12bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac12c1a0 con 0x7fa6ac071fc0 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac071170 con 0x7fa6ac0716a0 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b3fea700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac072670 con 0x7fa6ac12b7e0 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 0x7fa6ac12bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1d86700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 0x7fa6ac071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1585700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 0x7fa6ac12b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1d86700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 0x7fa6ac071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49648/0 (socket says 172.21.15.155:49648) 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 0x7fa6ac12bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52544/0 (socket says 172.21.15.155:52544) 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1585700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 0x7fa6ac12b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56692/0 (socket says 172.21.15.155:56692) 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1d86700 1 -- 172.21.15.155:0/2396518675 learned_addr learned my addr 172.21.15.155:0/2396518675 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 -- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 msgr2=0x7fa6ac071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 --2- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 0x7fa6ac071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 -- 172.21.15.155:0/2396518675 >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 msgr2=0x7fa6ac12b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 --2- 172.21.15.155:0/2396518675 >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 0x7fa6ac12b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b2587700 1 -- 172.21.15.155:0/2396518675 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0705c0 con 0x7fa6ac12b7e0 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1d86700 1 --2- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 0x7fa6ac071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.035+0000 7fa6b1585700 1 --2- 172.21.15.155:0/2396518675 >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 0x7fa6ac12b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b2587700 1 --2- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 0x7fa6ac12bc60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009d90 tx=0x7fa6a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0922e3e5808c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b0d84700 1 -- 172.21.15.155:0/2396518675 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400e070 con 0x7fa6ac12b7e0 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b0d84700 1 -- 172.21.15.155:0/2396518675 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a400bb20 con 0x7fa6ac12b7e0 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b0d84700 1 -- 172.21.15.155:0/2396518675 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4012430 con 0x7fa6ac12b7e0 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 -- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 msgr2=0x7fa6ac12bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 --2- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 0x7fa6ac12bc60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009d90 tx=0x7fa6a400b330 comp rx=0 tx=0).stop 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 -- 172.21.15.155:0/2396518675 shutdown_connections 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 --2- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac12b7e0 0x7fa6ac12bc60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 --2- 172.21.15.155:0/2396518675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac0716a0 0x7fa6ac071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 --2- 172.21.15.155:0/2396518675 >> v2:172.21.15.155:3300/0 conn(0x7fa6ac071fc0 0x7fa6ac12b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 -- 172.21.15.155:0/2396518675 >> 172.21.15.155:0/2396518675 conn(0x7fa6ac09dc90 msgr2=0x7fa6ac09e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:06.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 -- 172.21.15.155:0/2396518675 shutdown_connections 2024-04-25T11:03:06.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.036+0000 7fa6b3fea700 1 -- 172.21.15.155:0/2396518675 wait complete. 2024-04-25T11:03:06.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.037+0000 7fa6b3fea700 1 Processor -- start 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.037+0000 7fa6b3fea700 1 -- start start 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac0716a0 0x7fa6ac127100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ac071fc0 0x7fa6ac127640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac12b7e0 0x7fa6ac127b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac06cf80 con 0x7fa6ac071fc0 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac12ad30 con 0x7fa6ac12b7e0 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac08fe10 con 0x7fa6ac0716a0 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac0716a0 0x7fa6ac127100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac0716a0 0x7fa6ac127100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52552/0 (socket says 172.21.15.155:52552) 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b2587700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac12b7e0 0x7fa6ac127b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 -- 172.21.15.155:0/125796326 learned_addr learned my addr 172.21.15.155:0/125796326 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1585700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ac071fc0 0x7fa6ac127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 -- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac12b7e0 msgr2=0x7fa6ac127b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 --2- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac12b7e0 0x7fa6ac127b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 -- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ac071fc0 msgr2=0x7fa6ac127640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 --2- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ac071fc0 0x7fa6ac127640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 -- 172.21.15.155:0/125796326 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009550 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1585700 1 --2- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ac071fc0 0x7fa6ac127640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b2587700 1 --2- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ac12b7e0 0x7fa6ac127b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b1d86700 1 --2- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac0716a0 0x7fa6ac127100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00ec10 tx=0x7fa69c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00c850 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c00c9b0 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00cd40 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a80094a0 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.038+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a4009a70 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.039+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6ac06f7c0 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.039+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== mon.2 v2:172.21.15.155:3302/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa69c013470 con 0x7fa6ac0716a0 2024-04-25T11:03:06.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.040+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== mon.2 v2:172.21.15.155:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fa69c05c9b0 con 0x7fa6ac0716a0 2024-04-25T11:03:06.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.040+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 --> v1:172.21.15.155:6812/51677 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa690051a40 con 0x7fa69004dd70 2024-04-25T11:03:06.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.040+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== mon.2 v2:172.21.15.155:3302/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa69c0612c0 con 0x7fa6ac0716a0 2024-04-25T11:03:06.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.041+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== osd.3 v1:172.21.15.155:6812/51677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (unknown 0 0 2513694155) 0x7fa690051a40 con 0x7fa69004dd70 2024-04-25T11:03:06.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.054+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 --> v1:172.21.15.155:6812/51677 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6ac06c6b0 con 0x7fa69004dd70 2024-04-25T11:03:06.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6a27fc700 1 -- 172.21.15.155:0/125796326 <== osd.3 v1:172.21.15.155:6812/51677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7fa6ac06c6b0 con 0x7fa69004dd70 2024-04-25T11:03:06.054 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-25T11:03:06.055 INFO:teuthology.orchestra.run.smithi155.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:03:06.055 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 >> v1:172.21.15.155:6812/51677 conn(0x7fa69004dd70 legacy=0x7fa6900501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 >> v1:172.21.15.155:6816/49624 conn(0x7fa6900475f0 legacy=0x7fa690049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac0716a0 msgr2=0x7fa6ac127100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 --2- 172.21.15.155:0/125796326 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa6ac0716a0 0x7fa6ac127100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00ec10 tx=0x7fa69c00ef20 comp rx=0 tx=0).stop 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b2587700 1 -- 172.21.15.155:0/125796326 reap_dead start 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 shutdown_connections 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 >> 172.21.15.155:0/125796326 conn(0x7fa6ac09dc90 msgr2=0x7fa6ac0952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 shutdown_connections 2024-04-25T11:03:06.056 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.055+0000 7fa6b3fea700 1 -- 172.21.15.155:0/125796326 wait complete. 2024-04-25T11:03:06.066 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T11:03:06.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.188+0000 7f8e212a9700 1 Processor -- start 2024-04-25T11:03:06.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.188+0000 7f8e212a9700 1 -- start start 2024-04-25T11:03:06.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e212a9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e1c070ab0 0x7f8e1c070e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e212a9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c12b1a0 0x7f8e1c12b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e212a9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c06f440 con 0x7f8e1c0713d0 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e212a9700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c070240 con 0x7f8e1c070ab0 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e212a9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c070610 con 0x7f8e1c12b1a0 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1affd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e1c070ab0 0x7f8e1c070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1a7fc700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8e1c0713d0 0x7f8e1c12aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53536/0 (socket says 172.21.15.155:53536) 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1a7fc700 1 -- 172.21.15.155:0/921940682 learned_addr learned my addr 172.21.15.155:0/921940682 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1b7fe700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c12b1a0 0x7f8e1c12b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1affd700 1 -- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c12b1a0 msgr2=0x7f8e1c12b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1affd700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c12b1a0 0x7f8e1c12b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1affd700 1 -- 172.21.15.155:0/921940682 >> v1:172.21.15.155:6789/0 conn(0x7f8e1c0713d0 legacy=0x7f8e1c12aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1affd700 1 -- 172.21.15.155:0/921940682 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c06fa60 con 0x7f8e1c070ab0 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1b7fe700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c12b1a0 0x7f8e1c12b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e19ffb700 1 -- 172.21.15.155:0/921940682 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847602887 0 0) 0x7f8e1c06f440 con 0x7f8e1c0713d0 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.189+0000 7f8e1affd700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c070ab0 0x7f8e1c070e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009840 tx=0x7f8e0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d22799370359fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e19ffb700 1 -- 172.21.15.155:0/921940682 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c011830 con 0x7f8e1c070ab0 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e19ffb700 1 -- 172.21.15.155:0/921940682 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c011990 con 0x7f8e1c070ab0 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e19ffb700 1 -- 172.21.15.155:0/921940682 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c011d20 con 0x7f8e1c070ab0 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 -- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c070ab0 msgr2=0x7f8e1c070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c070ab0 0x7f8e1c070e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009840 tx=0x7f8e0c009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 -- 172.21.15.155:0/921940682 shutdown_connections 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c070ab0 0x7f8e1c070e90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 --2- 172.21.15.155:0/921940682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c12b1a0 0x7f8e1c12b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 -- 172.21.15.155:0/921940682 >> 172.21.15.155:0/921940682 conn(0x7f8e1c099b70 msgr2=0x7f8e1c0972d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:06.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 -- 172.21.15.155:0/921940682 shutdown_connections 2024-04-25T11:03:06.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.190+0000 7f8e212a9700 1 -- 172.21.15.155:0/921940682 wait complete. 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.191+0000 7f8e212a9700 1 Processor -- start 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.191+0000 7f8e212a9700 1 -- start start 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e212a9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 0x7f8e1c08b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e212a9700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e1c0713d0 0x7f8e1c08b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e212a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c12b1a0 0x7f8e1c1bea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e212a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c08c360 con 0x7f8e1c12b1a0 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e212a9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c12ad20 con 0x7f8e1c070ab0 2024-04-25T11:03:06.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e212a9700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c08c860 con 0x7f8e1c0713d0 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 0x7f8e1c08b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c12b1a0 0x7f8e1c1bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 0x7f8e1c08b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49672/0 (socket says 172.21.15.155:49672) 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 -- 172.21.15.155:0/3866550117 learned_addr learned my addr 172.21.15.155:0/3866550117 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1a7fc700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e1c0713d0 0x7f8e1c08b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 -- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e1c0713d0 msgr2=0x7f8e1c08b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e1c0713d0 0x7f8e1c08b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 -- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c12b1a0 msgr2=0x7f8e1c1bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c12b1a0 0x7f8e1c1bea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 -- 172.21.15.155:0/3866550117 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14009a70 con 0x7f8e1c070ab0 2024-04-25T11:03:06.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1b7fe700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c12b1a0 0x7f8e1c1bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.192+0000 7f8e1affd700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 0x7f8e1c08b270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009fd0 tx=0x7f8e0c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.193+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0043c0 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.193+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c01f070 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.193+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c0094a0 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.193+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0036a0 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.193+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c08d0e0 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.194+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c062510 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.194+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f8e0c021030 con 0x7f8e1c070ab0 2024-04-25T11:03:06.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.194+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f8e0c05d930 con 0x7f8e1c070ab0 2024-04-25T11:03:06.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.198+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f8e0c03f010 con 0x7f8e1c070ab0 2024-04-25T11:03:06.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.357+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e1c0679e0 con 0x7f8e080475f0 2024-04-25T11:03:06.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.357+0000 7f8e037fe700 1 -- 172.21.15.155:0/3866550117 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7f8e1c0679e0 con 0x7f8e080475f0 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 >> v1:172.21.15.155:6816/49624 conn(0x7f8e080475f0 legacy=0x7f8e08049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 msgr2=0x7f8e1c08b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 0x7f8e1c08b270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009fd0 tx=0x7f8e0c004750 comp rx=0 tx=0).stop 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 shutdown_connections 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e1c12b1a0 0x7f8e1c1bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e1c070ab0 0x7f8e1c08b270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 --2- 172.21.15.155:0/3866550117 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f8e1c0713d0 0x7f8e1c08b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 >> 172.21.15.155:0/3866550117 conn(0x7f8e1c099b70 msgr2=0x7f8e1c1239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 shutdown_connections 2024-04-25T11:03:06.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.359+0000 7f8e212a9700 1 -- 172.21.15.155:0/3866550117 wait complete. 2024-04-25T11:03:06.360 INFO:teuthology.orchestra.run.smithi155.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T11:03:06.370 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 Processor -- start 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 -- start start 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c12bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12b790 0x7f0d3c1286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c070fc0 con 0x7f0d3c12c200 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0703a0 con 0x7f0d3c12b790 2024-04-25T11:03:06.495 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.493+0000 7f0d422a1700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c070c80 con 0x7f0d3c06f4a0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d4083e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12b790 0x7f0d3c1286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3b7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0d3c12c200 0x7f0d3c127fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53540/0 (socket says 172.21.15.155:53540) 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3bfff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c12bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3b7fe700 1 -- 172.21.15.155:0/2615382026 learned_addr learned my addr 172.21.15.155:0/2615382026 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3affd700 1 -- 172.21.15.155:0/2615382026 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038008562 0 0) 0x7f0d3c070fc0 con 0x7f0d3c12c200 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3bfff700 1 -- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12b790 msgr2=0x7f0d3c1286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3bfff700 1 --2- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12b790 0x7f0d3c1286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3bfff700 1 -- 172.21.15.155:0/2615382026 >> v1:172.21.15.155:6789/0 conn(0x7f0d3c12c200 legacy=0x7f0d3c127fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3bfff700 1 -- 172.21.15.155:0/2615382026 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c06f880 con 0x7f0d3c06f4a0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d4083e700 1 --2- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12b790 0x7f0d3c1286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3bfff700 1 --2- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c12bcc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009870 tx=0x7f0d30009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a36a5ecaab4aa76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3affd700 1 -- 172.21.15.155:0/2615382026 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011830 con 0x7f0d3c06f4a0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3affd700 1 -- 172.21.15.155:0/2615382026 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30011990 con 0x7f0d3c06f4a0 2024-04-25T11:03:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.494+0000 7f0d3affd700 1 -- 172.21.15.155:0/2615382026 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011d20 con 0x7f0d3c06f4a0 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 -- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 msgr2=0x7f0d3c12bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 --2- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c12bcc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009870 tx=0x7f0d30009b80 comp rx=0 tx=0).stop 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 -- 172.21.15.155:0/2615382026 shutdown_connections 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 --2- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c12bcc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 --2- 172.21.15.155:0/2615382026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12b790 0x7f0d3c1286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 -- 172.21.15.155:0/2615382026 >> 172.21.15.155:0/2615382026 conn(0x7f0d3c09dbc0 msgr2=0x7f0d3c09dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 -- 172.21.15.155:0/2615382026 shutdown_connections 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.495+0000 7f0d422a1700 1 -- 172.21.15.155:0/2615382026 wait complete. 2024-04-25T11:03:06.497 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 Processor -- start 2024-04-25T11:03:06.499 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 -- start start 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c08b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c12b790 0x7f0d3c08b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 0x7f0d3c08ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c127f50 con 0x7f0d3c12b790 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c08c430 con 0x7f0d3c12c200 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d422a1700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c125760 con 0x7f0d3c06f4a0 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d3bfff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c08b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d4083e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 0x7f0d3c08ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d3b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c12b790 0x7f0d3c08b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d4083e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 0x7f0d3c08ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:49688/0 (socket says 172.21.15.155:49688) 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d3bfff700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c08b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:52576/0 (socket says 172.21.15.155:52576) 2024-04-25T11:03:06.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.496+0000 7f0d4083e700 1 -- 172.21.15.155:0/3377688364 learned_addr learned my addr 172.21.15.155:0/3377688364 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d4083e700 1 -- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 msgr2=0x7f0d3c08b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d4083e700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c08b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d4083e700 1 -- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c12b790 msgr2=0x7f0d3c08b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d4083e700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c12b790 0x7f0d3c08b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d4083e700 1 -- 172.21.15.155:0/3377688364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d34009a70 con 0x7f0d3c12c200 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d3b7fe700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c12b790 0x7f0d3c08b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d3bfff700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c08b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d4083e700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 0x7f0d3c08ba80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400efc0 tx=0x7f0d3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400ca90 con 0x7f0d3c12c200 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3400f040 con 0x7f0d3c12c200 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d300094a0 con 0x7f0d3c12c200 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400ce10 con 0x7f0d3c12c200 2024-04-25T11:03:06.501 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.497+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c08cbe0 con 0x7f0d3c12c200 2024-04-25T11:03:06.502 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.498+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c08d7c0 con 0x7f0d3c12c200 2024-04-25T11:03:06.502 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.498+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mon.1 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0d34014480 con 0x7f0d3c12c200 2024-04-25T11:03:06.502 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.498+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mon.1 v2:172.21.15.155:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f0d3405cea0 con 0x7f0d3c12c200 2024-04-25T11:03:06.502 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.502+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mon.1 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f0d3403a920 con 0x7f0d3c12c200 2024-04-25T11:03:06.656 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.657+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d3c062510 con 0x7f0d280475f0 2024-04-25T11:03:06.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.658+0000 7f0d38ff9700 1 -- 172.21.15.155:0/3377688364 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34589 (unknown 2965378022 0 4146391071) 0x7f0d3c062510 con 0x7f0d280475f0 2024-04-25T11:03:06.658 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 >> v1:172.21.15.155:6816/49624 conn(0x7f0d280475f0 legacy=0x7f0d28049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 msgr2=0x7f0d3c08ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 0x7f0d3c08ba80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400efc0 tx=0x7f0d3400c3a0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 shutdown_connections 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c12c200 0x7f0d3c08ba80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f0d3c06f4a0 0x7f0d3c08b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 --2- 172.21.15.155:0/3377688364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c12b790 0x7f0d3c08b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 >> 172.21.15.155:0/3377688364 conn(0x7f0d3c09dbc0 msgr2=0x7f0d3c0976b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 shutdown_connections 2024-04-25T11:03:06.660 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:06.660+0000 7f0d422a1700 1 -- 172.21.15.155:0/3377688364 wait complete. 2024-04-25T11:03:06.661 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:06.671 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-04-25T11:03:05.993190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110980,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376380,"statfs":{"total":386547056640,"available":386433413120,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":327696,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":65,"num_write":90,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.769284"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224548+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:57.224548+0000","last_peered":"2024-04-25T11:02:57.224548+0000","last_clean":"2024-04-25T11:02:57.224548+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:57.224548+0000","last_undegraded":"2024-04-25T11:02:57.224548+0000","last_fullsized":"2024-04-25T11:02:57.224548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:33.129142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226911+0000","last_change":"2024-04-25T11:02:52.039896+0000","last_active":"2024-04-25T11:02:57.226911+0000","last_peered":"2024-04-25T11:02:57.226911+0000","last_clean":"2024-04-25T11:02:57.226911+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:02:57.226911+0000","last_undegraded":"2024-04-25T11:02:57.226911+0000","last_fullsized":"2024-04-25T11:02:57.226911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.972806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224526+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:57.224526+0000","last_peered":"2024-04-25T11:02:57.224526+0000","last_clean":"2024-04-25T11:02:57.224526+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:57.224526+0000","last_undegraded":"2024-04-25T11:02:57.224526+0000","last_fullsized":"2024-04-25T11:02:57.224526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:24.325806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:23.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":157,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226905+0000","last_change":"2024-04-25T11:02:52.039856+0000","last_active":"2024-04-25T11:02:57.226905+0000","last_peered":"2024-04-25T11:02:57.226905+0000","last_clean":"2024-04-25T11:02:57.226905+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:02:57.226905+0000","last_undegraded":"2024-04-25T11:02:57.226905+0000","last_fullsized":"2024-04-25T11:02:57.226905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:58.405672+0000","last_clean_scrub_stamp":"2024-04-25T11:00:58.405672+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224458+0000","last_change":"2024-04-25T11:02:54.049124+0000","last_active":"2024-04-25T11:02:57.224458+0000","last_peered":"2024-04-25T11:02:57.224458+0000","last_clean":"2024-04-25T11:02:57.224458+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:02:57.224458+0000","last_undegraded":"2024-04-25T11:02:57.224458+0000","last_fullsized":"2024-04-25T11:02:57.224458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:18.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T11:02:56.752416+0000","last_change":"2024-04-25T11:02:54.048580+0000","last_active":"2024-04-25T11:02:56.752416+0000","last_peered":"2024-04-25T11:02:56.752416+0000","last_clean":"2024-04-25T11:02:56.752416+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:02:56.752416+0000","last_undegraded":"2024-04-25T11:02:56.752416+0000","last_fullsized":"2024-04-25T11:02:56.752416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:35.802506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.224431+0000","last_change":"2024-04-25T11:02:54.268917+0000","last_active":"2024-04-25T11:02:57.224431+0000","last_peered":"2024-04-25T11:02:57.224431+0000","last_clean":"2024-04-25T11:02:57.224431+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:02:57.224431+0000","last_undegraded":"2024-04-25T11:02:57.224431+0000","last_fullsized":"2024-04-25T11:02:57.224431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.092289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-25T11:02:57.226923+0000","last_change":"2024-04-25T11:02:52.040177+0000","last_active":"2024-04-25T11:02:57.226923+0000","last_peered":"2024-04-25T11:02:57.226923+0000","last_clean":"2024-04-25T11:02:57.226923+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:02:57.226923+0000","last_undegraded":"2024-04-25T11:02:57.226923+0000","last_fullsized":"2024-04-25T11:02:57.226923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:54.647842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182404,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]}]},{"osd":2,"up_from":24,"seq":103079215108,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":0,"up_from":22,"seq":94489280517,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":1,"up_from":23,"seq":98784247813,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:06.675 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.675 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.675 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.675 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.675 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T11:00:58.405672+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=58, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.676 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.676 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.676 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.676 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:06.676 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:03:26.676 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:26.812 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 Processor -- start 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 -- start start 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c129450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c129990 0x7f657c12c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 0x7f657c12cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c072410 con 0x7f657c12cab0 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c067130 con 0x7f657c129990 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f65831d5700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f657c071b40 con 0x7f657c129070 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f6581772700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 0x7f657c12cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f657bfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c129990 0x7f657c12c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f6580f71700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c129450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f6581772700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 0x7f657c12cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:38750/0 (socket says 172.21.15.155:38750) 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.810+0000 7f6581772700 1 -- 172.21.15.155:0/3767818709 learned_addr learned my addr 172.21.15.155:0/3767818709 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657bfff700 1 -- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 msgr2=0x7f657c129450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:26.813 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657bfff700 1 --2- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c129450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657bfff700 1 -- 172.21.15.155:0/3767818709 >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 msgr2=0x7f657c12cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657bfff700 1 --2- 172.21.15.155:0/3767818709 >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 0x7f657c12cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657bfff700 1 -- 172.21.15.155:0/3767818709 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c12d450 con 0x7f657c129990 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f6581772700 1 --2- 172.21.15.155:0/3767818709 >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 0x7f657c12cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f6580f71700 1 --2- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c129450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657bfff700 1 --2- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c129990 0x7f657c12c570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6568009840 tx=0x7f6568009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b27a2b7d1b944d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657b7fe700 1 -- 172.21.15.155:0/3767818709 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568011830 con 0x7f657c129990 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657b7fe700 1 -- 172.21.15.155:0/3767818709 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568011990 con 0x7f657c129990 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.811+0000 7f657b7fe700 1 -- 172.21.15.155:0/3767818709 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568011d20 con 0x7f657c129990 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 -- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c129990 msgr2=0x7f657c12c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 --2- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c129990 0x7f657c12c570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6568009840 tx=0x7f6568009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 -- 172.21.15.155:0/3767818709 shutdown_connections 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 --2- 172.21.15.155:0/3767818709 >> v2:172.21.15.155:3300/0 conn(0x7f657c12cab0 0x7f657c12cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.814 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 --2- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c129450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 --2- 172.21.15.155:0/3767818709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c129990 0x7f657c12c570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 -- 172.21.15.155:0/3767818709 >> 172.21.15.155:0/3767818709 conn(0x7f657c09ee80 msgr2=0x7f657c09f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:26.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 -- 172.21.15.155:0/3767818709 shutdown_connections 2024-04-25T11:03:26.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 -- 172.21.15.155:0/3767818709 wait complete. 2024-04-25T11:03:26.817 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.812+0000 7f65831d5700 1 Processor -- start 2024-04-25T11:03:26.817 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 -- start start 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c1c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 0x7f657c1c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c12cab0 0x7f657c1bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c1286e0 con 0x7f657c129990 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c132f20 con 0x7f657c12cab0 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f65831d5700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f657c131eb0 con 0x7f657c129070 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 0x7f657c1c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f6581772700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c12cab0 0x7f657c1bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 0x7f657c1c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:38762/0 (socket says 172.21.15.155:38762) 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 -- 172.21.15.155:0/3214211759 learned_addr learned my addr 172.21.15.155:0/3214211759 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f6580f71700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c1c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 -- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 msgr2=0x7f657c1c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:26.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c1c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 -- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c12cab0 msgr2=0x7f657c1bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c12cab0 0x7f657c1bfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f657bfff700 1 -- 172.21.15.155:0/3214211759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700094a0 con 0x7f657c129990 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.813+0000 7f6581772700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c12cab0 0x7f657c1bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.814+0000 7f657bfff700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 0x7f657c1c36d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65680090b0 tx=0x7f6568004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.814+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568004240 con 0x7f657c129990 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.814+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656801f070 con 0x7f657c129990 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.814+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680036a0 con 0x7f657c129990 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.814+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6574009b30 con 0x7f657c129990 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.814+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65680094a0 con 0x7f657c129990 2024-04-25T11:03:26.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.815+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c128c30 con 0x7f657c129990 2024-04-25T11:03:26.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.815+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f6568021030 con 0x7f657c129990 2024-04-25T11:03:26.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.815+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f656805d670 con 0x7f657c129990 2024-04-25T11:03:26.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.819+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f65680253a0 con 0x7f657c129990 2024-04-25T11:03:26.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.977+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f657c129df0 con 0x7f65600475f0 2024-04-25T11:03:26.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.978+0000 7f65797fa700 1 -- 172.21.15.155:0/3214211759 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34599 (unknown 2965378022 0 1084849129) 0x7f6570009820 con 0x7f65600475f0 2024-04-25T11:03:26.979 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 >> v1:172.21.15.155:6816/49624 conn(0x7f65600475f0 legacy=0x7f6560049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 msgr2=0x7f657c1c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 0x7f657c1c36d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65680090b0 tx=0x7f6568004750 comp rx=0 tx=0).stop 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 shutdown_connections 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f657c12cab0 0x7f657c1bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f657c129070 0x7f657c1c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 --2- 172.21.15.155:0/3214211759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f657c129990 0x7f657c1c36d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:26.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 >> 172.21.15.155:0/3214211759 conn(0x7f657c09ee80 msgr2=0x7f657c0952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:26.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 shutdown_connections 2024-04-25T11:03:26.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:26.980+0000 7f65831d5700 1 -- 172.21.15.155:0/3214211759 wait complete. 2024-04-25T11:03:26.982 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:26.992 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T11:03:25.997142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110948,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376412,"statfs":{"total":386547056640,"available":386433445888,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002421"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:02:58.234460+0000","last_change":"2024-04-25T11:02:54.049531+0000","last_active":"2024-04-25T11:02:58.234460+0000","last_peered":"2024-04-25T11:02:58.234460+0000","last_clean":"2024-04-25T11:02:58.234460+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:02:58.234460+0000","last_undegraded":"2024-04-25T11:02:58.234460+0000","last_fullsized":"2024-04-25T11:02:58.234460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:07.339158+0000","last_change":"2024-04-25T11:03:07.339286+0000","last_active":"2024-04-25T11:03:07.339158+0000","last_peered":"2024-04-25T11:03:07.339158+0000","last_clean":"2024-04-25T11:03:07.339158+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:03:07.339158+0000","last_undegraded":"2024-04-25T11:03:07.339158+0000","last_fullsized":"2024-04-25T11:03:07.339158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:07.339133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:07.339133+0000","last_clean_scrub_stamp":"2024-04-25T11:03:07.339133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:18.686538+0000","scrub_duration":0.0015867190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:02:58.234519+0000","last_change":"2024-04-25T11:02:54.049501+0000","last_active":"2024-04-25T11:02:58.234519+0000","last_peered":"2024-04-25T11:02:58.234519+0000","last_clean":"2024-04-25T11:02:58.234519+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:02:58.234519+0000","last_undegraded":"2024-04-25T11:02:58.234519+0000","last_fullsized":"2024-04-25T11:02:58.234519+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:02:58.234229+0000","last_change":"2024-04-25T11:02:54.269805+0000","last_active":"2024-04-25T11:02:58.234229+0000","last_peered":"2024-04-25T11:02:58.234229+0000","last_clean":"2024-04-25T11:02:58.234229+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:02:58.234229+0000","last_undegraded":"2024-04-25T11:02:58.234229+0000","last_fullsized":"2024-04-25T11:02:58.234229+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:59.411282+0000","last_clean_scrub_stamp":"2024-04-25T11:00:59.411282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":169,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:09.351464+0000","last_change":"2024-04-25T11:03:09.351620+0000","last_active":"2024-04-25T11:03:09.351464+0000","last_peered":"2024-04-25T11:03:09.351464+0000","last_clean":"2024-04-25T11:03:09.351464+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:03:09.351464+0000","last_undegraded":"2024-04-25T11:03:09.351464+0000","last_fullsized":"2024-04-25T11:03:09.351464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'97","last_scrub_stamp":"2024-04-25T11:03:09.351439+0000","last_deep_scrub":"26'97","last_deep_scrub_stamp":"2024-04-25T11:03:09.351439+0000","last_clean_scrub_stamp":"2024-04-25T11:03:09.351439+0000","objects_scrubbed":2,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:12.430212+0000","scrub_duration":0.0044096819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:10.363727+0000","last_change":"2024-04-25T11:03:10.363852+0000","last_active":"2024-04-25T11:03:10.363727+0000","last_peered":"2024-04-25T11:03:10.363727+0000","last_clean":"2024-04-25T11:03:10.363727+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:03:10.363727+0000","last_undegraded":"2024-04-25T11:03:10.363727+0000","last_fullsized":"2024-04-25T11:03:10.363727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:10.363700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:10.363700+0000","last_clean_scrub_stamp":"2024-04-25T11:03:10.363700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:11.908030+0000","scrub_duration":0.0015504449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:06.178352+0000","last_change":"2024-04-25T11:03:06.178459+0000","last_active":"2024-04-25T11:03:06.178352+0000","last_peered":"2024-04-25T11:03:06.178352+0000","last_clean":"2024-04-25T11:03:06.178352+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:03:06.178352+0000","last_undegraded":"2024-04-25T11:03:06.178352+0000","last_fullsized":"2024-04-25T11:03:06.178352+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:06.178331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:06.178331+0000","last_clean_scrub_stamp":"2024-04-25T11:03:06.178331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:34.276651+0000","scrub_duration":0.001388713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:14.284279+0000","last_change":"2024-04-25T11:03:14.284402+0000","last_active":"2024-04-25T11:03:14.284279+0000","last_peered":"2024-04-25T11:03:14.284279+0000","last_clean":"2024-04-25T11:03:14.284279+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:03:14.284279+0000","last_undegraded":"2024-04-25T11:03:14.284279+0000","last_fullsized":"2024-04-25T11:03:14.284279+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:14.284247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:14.284247+0000","last_clean_scrub_stamp":"2024-04-25T11:03:14.284247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:24.887700+0000","scrub_duration":0.001699213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:11.421747+0000","last_change":"2024-04-25T11:03:11.421891+0000","last_active":"2024-04-25T11:03:11.421747+0000","last_peered":"2024-04-25T11:03:11.421747+0000","last_clean":"2024-04-25T11:03:11.421747+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:03:11.421747+0000","last_undegraded":"2024-04-25T11:03:11.421747+0000","last_fullsized":"2024-04-25T11:03:11.421747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:11.421722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:11.421722+0000","last_clean_scrub_stamp":"2024-04-25T11:03:11.421722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:08.290002+0000","scrub_duration":0.0016094390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182408,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]}]},{"osd":2,"up_from":24,"seq":103079215112,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]}]},{"osd":0,"up_from":22,"seq":94489280521,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]},{"osd":1,"up_from":23,"seq":98784247817,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:26.993 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:26.993 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:26.994 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T11:00:59.411282+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=3, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:03:26.994 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:03:46.995 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:03:47.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 Processor -- start 2024-04-25T11:03:47.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 -- start start 2024-04-25T11:03:47.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 0x7fa9a412f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 0x7fa9a406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a4072440 con 0x7fa9a406dac0 2024-04-25T11:03:47.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a406fb70 con 0x7fa9a412edd0 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a990a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a406ffb0 con 0x7fa9a412f6f0 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a27fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 0x7fa9a406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a2ffd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 0x7fa9a412f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a37fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa9a406dac0 0x7fa9a406df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:47092/0 (socket says 172.21.15.155:47092) 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a2ffd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 0x7fa9a412f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:32930/0 (socket says 172.21.15.155:32930) 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a27fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 0x7fa9a406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41198/0 (socket says 172.21.15.155:41198) 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a37fe700 1 -- 172.21.15.155:0/3192602455 learned_addr learned my addr 172.21.15.155:0/3192602455 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a27fc700 1 -- 172.21.15.155:0/3192602455 >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 msgr2=0x7fa9a412f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a27fc700 1 --2- 172.21.15.155:0/3192602455 >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 0x7fa9a412f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.128+0000 7fa9a27fc700 1 -- 172.21.15.155:0/3192602455 >> v1:172.21.15.155:6789/0 conn(0x7fa9a406dac0 legacy=0x7fa9a406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a27fc700 1 -- 172.21.15.155:0/3192602455 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a406fd80 con 0x7fa9a412f6f0 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a1ffb700 1 -- 172.21.15.155:0/3192602455 <== mon.0 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318683780 0 0) 0x7fa9a4072440 con 0x7fa9a406dac0 2024-04-25T11:03:47.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a2ffd700 1 --2- 172.21.15.155:0/3192602455 >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 0x7fa9a412f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a27fc700 1 --2- 172.21.15.155:0/3192602455 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 0x7fa9a406d580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa998009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d92d233ce0317ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a1ffb700 1 -- 172.21.15.155:0/3192602455 <== mon.1 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011830 con 0x7fa9a412f6f0 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a1ffb700 1 -- 172.21.15.155:0/3192602455 <== mon.1 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998011990 con 0x7fa9a412f6f0 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a1ffb700 1 -- 172.21.15.155:0/3192602455 <== mon.1 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011d20 con 0x7fa9a412f6f0 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 -- 172.21.15.155:0/3192602455 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 msgr2=0x7fa9a406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 --2- 172.21.15.155:0/3192602455 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 0x7fa9a406d580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa998009b50 comp rx=0 tx=0).stop 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 -- 172.21.15.155:0/3192602455 shutdown_connections 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 --2- 172.21.15.155:0/3192602455 >> v2:172.21.15.155:3300/0 conn(0x7fa9a412edd0 0x7fa9a412f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 --2- 172.21.15.155:0/3192602455 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412f6f0 0x7fa9a406d580 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 -- 172.21.15.155:0/3192602455 >> 172.21.15.155:0/3192602455 conn(0x7fa9a409e9f0 msgr2=0x7fa9a409ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.129+0000 7fa9a990a700 1 -- 172.21.15.155:0/3192602455 shutdown_connections 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.130+0000 7fa9a990a700 1 -- 172.21.15.155:0/3192602455 wait complete. 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.130+0000 7fa9a990a700 1 Processor -- start 2024-04-25T11:03:47.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 -- start start 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 0x7fa9a40906c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412edd0 0x7fa9a4090c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9a412f6f0 0x7fa9a4091140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4092e00 con 0x7fa9a406dac0 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4093a30 con 0x7fa9a412edd0 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a990a700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4092a10 con 0x7fa9a412f6f0 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 0x7fa9a40906c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 0x7fa9a40906c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:32944/0 (socket says 172.21.15.155:32944) 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 -- 172.21.15.155:0/203744333 learned_addr learned my addr 172.21.15.155:0/203744333 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a37fe700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9a412f6f0 0x7fa9a4091140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a27fc700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412edd0 0x7fa9a4090c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 -- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9a412f6f0 msgr2=0x7fa9a4091140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.136 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9a412f6f0 0x7fa9a4091140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 -- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412edd0 msgr2=0x7fa9a4090c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412edd0 0x7fa9a4090c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 -- 172.21.15.155:0/203744333 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940094a0 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a27fc700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412edd0 0x7fa9a4090c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.131+0000 7fa9a2ffd700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 0x7fa9a40906c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa99400ebe0 tx=0x7fa99400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.132+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400c850 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.132+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99400c9b0 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.132+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9980094a0 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.132+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400cd40 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.132+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a41c30c0 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.133+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a409f6f0 con 0x7fa9a406dac0 2024-04-25T11:03:47.137 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.133+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa994013470 con 0x7fa9a406dac0 2024-04-25T11:03:47.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.133+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7fa99405c8e0 con 0x7fa9a406dac0 2024-04-25T11:03:47.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.137+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7fa99403a530 con 0x7fa9a406dac0 2024-04-25T11:03:47.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.292+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 --> v1:172.21.15.155:6816/49624 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9a406f400 con 0x7fa9900475f0 2024-04-25T11:03:47.294 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.294+0000 7fa98bfff700 1 -- 172.21.15.155:0/203744333 <== mgr.14112 v1:172.21.15.155:6816/49624 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34727 (unknown 2965378022 0 2852107129) 0x7fa998011630 con 0x7fa9900475f0 2024-04-25T11:03:47.295 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T11:03:47.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 >> v1:172.21.15.155:6816/49624 conn(0x7fa9900475f0 legacy=0x7fa990049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 msgr2=0x7fa9a40906c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 0x7fa9a40906c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa99400ebe0 tx=0x7fa99400eef0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 shutdown_connections 2024-04-25T11:03:47.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7fa9a412f6f0 0x7fa9a4091140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a406dac0 0x7fa9a40906c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 --2- 172.21.15.155:0/203744333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a412edd0 0x7fa9a4090c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 >> 172.21.15.155:0/203744333 conn(0x7fa9a409e9f0 msgr2=0x7fa9a409acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:47.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 shutdown_connections 2024-04-25T11:03:47.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.295+0000 7fa9a990a700 1 -- 172.21.15.155:0/203744333 wait complete. 2024-04-25T11:03:47.297 INFO:teuthology.orchestra.run.smithi155.stderr:dumped all 2024-04-25T11:03:47.308 INFO:teuthology.orchestra.run.smithi155.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-25T11:03:46.001694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110948,"kb_used_data":2116,"kb_used_omap":4,"kb_used_meta":108795,"kb_avail":377376412,"statfs":{"total":386547056640,"available":386433445888,"internally_reserved":0,"allocated":2166784,"data_stored":1525827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":111406564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002721"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:34.283328+0000","last_change":"2024-04-25T11:03:34.283448+0000","last_active":"2024-04-25T11:03:34.283328+0000","last_peered":"2024-04-25T11:03:34.283328+0000","last_clean":"2024-04-25T11:03:34.283328+0000","last_became_active":"2024-04-25T11:02:54.049244+0000","last_became_peered":"2024-04-25T11:02:54.049244+0000","last_unstale":"2024-04-25T11:03:34.283328+0000","last_undegraded":"2024-04-25T11:03:34.283328+0000","last_fullsized":"2024-04-25T11:03:34.283328+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:34.283303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:34.283303+0000","last_clean_scrub_stamp":"2024-04-25T11:03:34.283303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:13.759534+0000","scrub_duration":0.001547147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:07.339158+0000","last_change":"2024-04-25T11:03:07.339286+0000","last_active":"2024-04-25T11:03:07.339158+0000","last_peered":"2024-04-25T11:03:07.339158+0000","last_clean":"2024-04-25T11:03:07.339158+0000","last_became_active":"2024-04-25T11:02:52.039666+0000","last_became_peered":"2024-04-25T11:02:52.039666+0000","last_unstale":"2024-04-25T11:03:07.339158+0000","last_undegraded":"2024-04-25T11:03:07.339158+0000","last_fullsized":"2024-04-25T11:03:07.339158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:07.339133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:07.339133+0000","last_clean_scrub_stamp":"2024-04-25T11:03:07.339133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:18.686538+0000","scrub_duration":0.0015867190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:26.188386+0000","last_change":"2024-04-25T11:03:26.188496+0000","last_active":"2024-04-25T11:03:26.188386+0000","last_peered":"2024-04-25T11:03:26.188386+0000","last_clean":"2024-04-25T11:03:26.188386+0000","last_became_active":"2024-04-25T11:02:54.049246+0000","last_became_peered":"2024-04-25T11:02:54.049246+0000","last_unstale":"2024-04-25T11:03:26.188386+0000","last_undegraded":"2024-04-25T11:03:26.188386+0000","last_fullsized":"2024-04-25T11:03:26.188386+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:26.188362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:26.188362+0000","last_clean_scrub_stamp":"2024-04-25T11:03:26.188362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:58.420874+0000","scrub_duration":0.001564654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:31.179837+0000","last_change":"2024-04-25T11:03:31.179980+0000","last_active":"2024-04-25T11:03:31.179837+0000","last_peered":"2024-04-25T11:03:31.179837+0000","last_clean":"2024-04-25T11:03:31.179837+0000","last_became_active":"2024-04-25T11:02:54.269555+0000","last_became_peered":"2024-04-25T11:02:54.269555+0000","last_unstale":"2024-04-25T11:03:31.179837+0000","last_undegraded":"2024-04-25T11:03:31.179837+0000","last_fullsized":"2024-04-25T11:03:31.179837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:31.179812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:31.179812+0000","last_clean_scrub_stamp":"2024-04-25T11:03:31.179812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:50.260476+0000","scrub_duration":0.001694636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"26'97","reported_seq":169,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:09.351464+0000","last_change":"2024-04-25T11:03:09.351620+0000","last_active":"2024-04-25T11:03:09.351464+0000","last_peered":"2024-04-25T11:03:09.351464+0000","last_clean":"2024-04-25T11:03:09.351464+0000","last_became_active":"2024-04-25T11:02:52.039555+0000","last_became_peered":"2024-04-25T11:02:52.039555+0000","last_unstale":"2024-04-25T11:03:09.351464+0000","last_undegraded":"2024-04-25T11:03:09.351464+0000","last_fullsized":"2024-04-25T11:03:09.351464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'97","last_scrub_stamp":"2024-04-25T11:03:09.351439+0000","last_deep_scrub":"26'97","last_deep_scrub_stamp":"2024-04-25T11:03:09.351439+0000","last_clean_scrub_stamp":"2024-04-25T11:03:09.351439+0000","objects_scrubbed":2,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:12.430212+0000","scrub_duration":0.0044096819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:10.363727+0000","last_change":"2024-04-25T11:03:10.363852+0000","last_active":"2024-04-25T11:03:10.363727+0000","last_peered":"2024-04-25T11:03:10.363727+0000","last_clean":"2024-04-25T11:03:10.363727+0000","last_became_active":"2024-04-25T11:02:54.048910+0000","last_became_peered":"2024-04-25T11:02:54.048910+0000","last_unstale":"2024-04-25T11:03:10.363727+0000","last_undegraded":"2024-04-25T11:03:10.363727+0000","last_fullsized":"2024-04-25T11:03:10.363727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:10.363700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:10.363700+0000","last_clean_scrub_stamp":"2024-04-25T11:03:10.363700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:11.908030+0000","scrub_duration":0.0015504449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:06.178352+0000","last_change":"2024-04-25T11:03:06.178459+0000","last_active":"2024-04-25T11:03:06.178352+0000","last_peered":"2024-04-25T11:03:06.178352+0000","last_clean":"2024-04-25T11:03:06.178352+0000","last_became_active":"2024-04-25T11:02:54.048324+0000","last_became_peered":"2024-04-25T11:02:54.048324+0000","last_unstale":"2024-04-25T11:03:06.178352+0000","last_undegraded":"2024-04-25T11:03:06.178352+0000","last_fullsized":"2024-04-25T11:03:06.178352+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:06.178331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:06.178331+0000","last_clean_scrub_stamp":"2024-04-25T11:03:06.178331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:34.276651+0000","scrub_duration":0.001388713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:14.284279+0000","last_change":"2024-04-25T11:03:14.284402+0000","last_active":"2024-04-25T11:03:14.284279+0000","last_peered":"2024-04-25T11:03:14.284279+0000","last_clean":"2024-04-25T11:03:14.284279+0000","last_became_active":"2024-04-25T11:02:54.268677+0000","last_became_peered":"2024-04-25T11:02:54.268677+0000","last_unstale":"2024-04-25T11:03:14.284279+0000","last_undegraded":"2024-04-25T11:03:14.284279+0000","last_fullsized":"2024-04-25T11:03:14.284279+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:14.284247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:14.284247+0000","last_clean_scrub_stamp":"2024-04-25T11:03:14.284247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:24.887700+0000","scrub_duration":0.001699213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:03:11.421747+0000","last_change":"2024-04-25T11:03:11.421891+0000","last_active":"2024-04-25T11:03:11.421747+0000","last_peered":"2024-04-25T11:03:11.421747+0000","last_clean":"2024-04-25T11:03:11.421747+0000","last_became_active":"2024-04-25T11:02:52.039601+0000","last_became_peered":"2024-04-25T11:02:52.039601+0000","last_unstale":"2024-04-25T11:03:11.421747+0000","last_undegraded":"2024-04-25T11:03:11.421747+0000","last_fullsized":"2024-04-25T11:03:11.421747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:03:11.421722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:03:11.421722+0000","last_clean_scrub_stamp":"2024-04-25T11:03:11.421722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:08.290002+0000","scrub_duration":0.0016094390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182412,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]},{"osd":2,"up_from":24,"seq":103079215116,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":245760,"data_stored":86540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":0,"up_from":22,"seq":94489280525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":835584,"data_stored":675839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":1,"up_from":23,"seq":98784247821,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27198,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":839680,"data_stored":676908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1159,"internal_metadata":27851641},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:03:47.309 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.427+0000 7f848ea71700 1 Processor -- start 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.427+0000 7f848ea71700 1 -- start start 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848ea71700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 0x7f8488071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848ea71700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071ba0 0x7f848812b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848ea71700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f848812bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848ea71700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8488070fc0 con 0x7f8488071280 2024-04-25T11:03:47.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848ea71700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880726c0 con 0x7f8488071ba0 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848ea71700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84880706f0 con 0x7f848812b870 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f8487fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071ba0 0x7f848812b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f848812bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848c80d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 0x7f8488071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848c80d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 0x7f8488071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:32956/0 (socket says 172.21.15.155:32956) 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f848812bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:36496/0 (socket says 172.21.15.155:36496) 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f8487fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071ba0 0x7f848812b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:41208/0 (socket says 172.21.15.155:41208) 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848c80d700 1 -- 172.21.15.155:0/926385557 learned_addr learned my addr 172.21.15.155:0/926385557 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 -- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071ba0 msgr2=0x7f848812b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 --2- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071ba0 0x7f848812b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 -- 172.21.15.155:0/926385557 >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 msgr2=0x7f8488071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 --2- 172.21.15.155:0/926385557 >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 0x7f8488071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 -- 172.21.15.155:0/926385557 --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848812c210 con 0x7f848812b870 2024-04-25T11:03:47.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848c80d700 1 --2- 172.21.15.155:0/926385557 >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 0x7f8488071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f848d00e700 1 --2- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f848812bcd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8480009dc0 tx=0x7f848000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94645b03bf97beea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f84877fe700 1 -- 172.21.15.155:0/926385557 <== mon.2 v2:172.21.15.155:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000bce0 con 0x7f848812b870 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f84877fe700 1 -- 172.21.15.155:0/926385557 <== mon.2 v2:172.21.15.155:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848000be40 con 0x7f848812b870 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.428+0000 7f84877fe700 1 -- 172.21.15.155:0/926385557 <== mon.2 v2:172.21.15.155:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800116a0 con 0x7f848812b870 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 -- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 msgr2=0x7f848812bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 --2- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f848812bcd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8480009dc0 tx=0x7f848000b330 comp rx=0 tx=0).stop 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 -- 172.21.15.155:0/926385557 shutdown_connections 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 --2- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f848812bcd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 --2- 172.21.15.155:0/926385557 >> v2:172.21.15.155:3300/0 conn(0x7f8488071280 0x7f8488071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 --2- 172.21.15.155:0/926385557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071ba0 0x7f848812b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 -- 172.21.15.155:0/926385557 >> 172.21.15.155:0/926385557 conn(0x7f848809de70 msgr2=0x7f848809e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 -- 172.21.15.155:0/926385557 shutdown_connections 2024-04-25T11:03:47.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.429+0000 7f848ea71700 1 -- 172.21.15.155:0/926385557 wait complete. 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 Processor -- start 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 -- start start 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071280 0x7f84881265c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 0x7f8488126b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f8488127040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488128c60 con 0x7f8488071ba0 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848808bd20 con 0x7f8488071280 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848ea71700 1 -- --> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f848808b450 con 0x7f848812b870 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f8487fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 0x7f8488126b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848d00e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f8488127040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f8487fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 0x7f8488126b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:32958/0 (socket says 172.21.15.155:32958) 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f848d00e700 1 --2- >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f8488127040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3302/0 says I am v2:172.21.15.155:36512/0 (socket says 172.21.15.155:36512) 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.430+0000 7f8487fff700 1 -- 172.21.15.155:0/824908506 learned_addr learned my addr 172.21.15.155:0/824908506 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T11:03:47.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f848c80d700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071280 0x7f84881265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f8487fff700 1 -- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 msgr2=0x7f8488127040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f8487fff700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f8488127040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f8487fff700 1 -- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071280 msgr2=0x7f84881265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f8487fff700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071280 0x7f84881265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f8487fff700 1 -- 172.21.15.155:0/824908506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009550 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f848c80d700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071280 0x7f84881265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f848d00e700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f8488127040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f8487fff700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 0x7f8488126b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f847400e9b0 tx=0x7f847400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474009710 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847400ee50 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8480009a70 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400ca50 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.431+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8488127580 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.432+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8488062510 con 0x7f8488071ba0 2024-04-25T11:03:47.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.432+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 9) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f847400cbb0 con 0x7f8488071ba0 2024-04-25T11:03:47.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.433+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 2961+0+0 (secure 0 0 0) 0x7f847405c7b0 con 0x7f8488071ba0 2024-04-25T11:03:47.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.436+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+145500 (secure 0 0 0) 0x7f847403a460 con 0x7f8488071ba0 2024-04-25T11:03:47.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.596+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8488127800 con 0x7f8488071ba0 2024-04-25T11:03:47.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.602+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f847403a460 con 0x7f8488071ba0 2024-04-25T11:03:47.603 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.602+0000 7f84857fa700 1 -- 172.21.15.155:0/824908506 <== mon.0 v2:172.21.15.155:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8474017070 con 0x7f8488071ba0 2024-04-25T11:03:47.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 >> v1:172.21.15.155:6816/49624 conn(0x7f846c0475f0 legacy=0x7f846c049ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 msgr2=0x7f8488126b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:03:47.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 0x7f8488126b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f847400e9b0 tx=0x7f847400ecc0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 shutdown_connections 2024-04-25T11:03:47.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3302/0,v1:172.21.15.155:6791/0] conn(0x7f848812b870 0x7f8488127040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.607 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8488071280 0x7f84881265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.607 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 --2- 172.21.15.155:0/824908506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8488071ba0 0x7f8488126b00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:03:47.607 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 >> 172.21.15.155:0/824908506 conn(0x7f848809de70 msgr2=0x7f8488129360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:03:47.607 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 shutdown_connections 2024-04-25T11:03:47.607 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T11:03:47.605+0000 7f848ea71700 1 -- 172.21.15.155:0/824908506 wait complete. 2024-04-25T11:03:47.618 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T11:03:47.618 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T11:03:47.621 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:03:47.621 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:47.652 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:03:47.652 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:03:47.652 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:47.724 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:03:47.724 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:03:47.724 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:47.790 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:03:47.791 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:03:47.791 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:47.858 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:03:47.858 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T11:03:47.858 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:03:47.858 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:47.918 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:03:47.918 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:03:47.918 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:47.981 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:03:47.981 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T11:03:47.981 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:03:47.981 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:48.037 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:03:48.037 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:03:48.037 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:48.094 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:03:48.094 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:03:48.094 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:03:48.151 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:03:48.151 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T11:03:48.152 DEBUG:teuthology.orchestra.run.smithi155:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2024-04-25T11:03:48.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:03:48.238 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T11:03:48.444 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:03:48.445 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T11:03:48.543 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:03:48.543 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T11:03:48.629 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:03:48.629 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T11:03:48.710 INFO:tasks.ceph:Archiving mon data... 2024-04-25T11:03:48.777 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/data/mon.a.tgz 2024-04-25T11:03:48.777 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-25T11:03:48.797 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.DAjoabMbz4 2024-04-25T11:03:48.797 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DAjoabMbz4 2024-04-25T11:03:48.962 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.DAjoabMbz4 2024-04-25T11:03:49.051 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.DAjoabMbz4 is 268KB 2024-04-25T11:03:49.147 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.DAjoabMbz4 2024-04-25T11:03:49.164 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/data/mon.b.tgz 2024-04-25T11:03:49.165 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-25T11:03:49.221 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.OdYKoNidXG 2024-04-25T11:03:49.221 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OdYKoNidXG 2024-04-25T11:03:49.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.OdYKoNidXG 2024-04-25T11:03:49.438 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.OdYKoNidXG is 309KB 2024-04-25T11:03:49.509 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.OdYKoNidXG 2024-04-25T11:03:49.526 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/data/mon.c.tgz 2024-04-25T11:03:49.527 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-25T11:03:49.584 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.NUO3qt2JUr 2024-04-25T11:03:49.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NUO3qt2JUr 2024-04-25T11:03:49.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.NUO3qt2JUr 2024-04-25T11:03:49.801 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.NUO3qt2JUr is 309KB 2024-04-25T11:03:49.896 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.NUO3qt2JUr 2024-04-25T11:03:49.914 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T11:03:49.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:03:50.038 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:03:50.038 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T11:03:50.040 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/remote/smithi155/crash 2024-04-25T11:03:50.041 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:03:50.104 INFO:tasks.ceph:Compressing logs... 2024-04-25T11:03:50.104 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51007.log 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43744.log 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58820.log 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.43744.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51007.log.gz 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37014.log 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43744.log.gz 2024-04-25T11:03:50.194 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50102.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58820.log: /var/log/ceph/ceph.tmp-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48178.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37014.log.gz 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58820.log.gz 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49903.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50102.log: /var/log/ceph/ceph-client.admin.48178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50102.log.gz 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49903.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48178.log.gz 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57326.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49903.log.gz 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56998.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50809.log: /var/log/ceph/ceph-client.admin.57326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2024-04-25T11:03:50.195 INFO:teuthology.orchestra.run.smithi155.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- 89.1%/var/log/ceph/ceph-client.admin.50241.log 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57326.log.gz 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.56998.log: /var/log/ceph/ceph-client.admin.47773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56998.log.gz 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47597.log 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gz 2024-04-25T11:03:50.196 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50736.log 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49770.log: /var/log/ceph/ceph-client.admin.47597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50736.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47597.log.gz 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58477.log 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48637.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50736.log.gz 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50072.log 2024-04-25T11:03:50.197 INFO:teuthology.orchestra.run.smithi155.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2024-04-25T11:03:50.198 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57624.log 2024-04-25T11:03:50.198 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50072.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58477.log.gz 2024-04-25T11:03:50.198 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47381.log 2024-04-25T11:03:50.198 INFO:teuthology.orchestra.run.smithi155.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50072.log.gz/var/log/ceph/ceph-client.admin.57624.log: 2024-04-25T11:03:50.198 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57497.log 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.57624.log.gz 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47381.log.gz/var/log/ceph/ceph-client.admin.57497.log: 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50561.log 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.43897.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57497.log.gz 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T11:03:50.199 INFO:teuthology.orchestra.run.smithi155.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49900.log 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.2.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50561.log.gz 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51033.log 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49900.log.gz 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58134.log 2024-04-25T11:03:50.200 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51033.log: /var/log/ceph/ceph-client.admin.57092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56228.log 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51033.log.gz 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57990.log 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.56228.log: -- /var/log/ceph/ceph-client.admin.50932.log 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58134.log.gz 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58575.log 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56228.log.gz 2024-04-25T11:03:50.201 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50932.log: gzip -5 --verbose -- 88.5%/var/log/ceph/ceph-client.admin.58281.log 2024-04-25T11:03:50.202 INFO:teuthology.orchestra.run.smithi155.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57990.log.gz 2024-04-25T11:03:50.202 INFO:teuthology.orchestra.run.smithi155.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2024-04-25T11:03:50.202 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57623.log 2024-04-25T11:03:50.202 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58926.log 2024-04-25T11:03:50.202 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58281.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58575.log.gz 2024-04-25T11:03:50.202 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57623.log: 88.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.58281.log.gz--verbose 2024-04-25T11:03:50.203 INFO:teuthology.orchestra.run.smithi155.stderr:-- /var/log/ceph/ceph-client.admin.57350.log 2024-04-25T11:03:50.203 INFO:teuthology.orchestra.run.smithi155.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57623.log.gz 2024-04-25T11:03:50.203 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-04-25T11:03:50.203 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58926.log: /var/log/ceph/ceph-client.admin.57350.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:03:50.203 INFO:teuthology.orchestra.run.smithi155.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58926.log.gz 2024-04-25T11:03:50.203 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51676.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57350.log.gz 2024-04-25T11:03:50.204 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47646.log 2024-04-25T11:03:50.204 INFO:teuthology.orchestra.run.smithi155.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-04-25T11:03:50.204 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58624.log 2024-04-25T11:03:50.204 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50979.log 2024-04-25T11:03:50.204 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47646.log: /var/log/ceph/ceph-client.admin.58624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57201.log 2024-04-25T11:03:50.204 INFO:teuthology.orchestra.run.smithi155.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47646.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58624.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47450.log 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57201.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50979.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48322.log 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57201.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47450.log: 95.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47450.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48322.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48322.log.gz 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57399.log 2024-04-25T11:03:50.205 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57399.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57399.log.gz 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47548.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47548.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47548.log.gz 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49623.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50457.log 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49623.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49623.log.gz 2024-04-25T11:03:50.206 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50457.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50457.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57621.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50783.log: /var/log/ceph/ceph-client.admin.57621.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57621.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49898.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48296.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49898.log: /var/log/ceph/ceph-client.admin.48296.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49898.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48296.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58232.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51177.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58232.log.gz 2024-04-25T11:03:50.207 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58673.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51177.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48177.log: /var/log/ceph/ceph-client.admin.58673.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58673.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51569.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57767.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51569.log: /var/log/ceph/ceph-client.admin.57767.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51569.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57767.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51458.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.51458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58771.log 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51458.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58771.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58771.log.gz 2024-04-25T11:03:50.208 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58330.log 2024-04-25T11:03:50.209 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58016.log 2024-04-25T11:03:50.212 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58330.log: /var/log/ceph/ceph-client.admin.58016.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58330.log.gz 2024-04-25T11:03:50.212 INFO:teuthology.orchestra.run.smithi155.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58016.log.gz 2024-04-25T11:03:50.212 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58379.log 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58379.log: /var/log/ceph/ceph-client.admin.43694.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58379.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57622.log 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58428.log 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57622.log: /var/log/ceph/ceph-client.admin.58428.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57622.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58428.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50344.log 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57302.log 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50344.log: 88.0%/var/log/ceph/ceph-client.admin.57302.log: -- replaced with /var/log/ceph/ceph-client.admin.50344.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57302.log.gz 2024-04-25T11:03:50.213 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44020.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49716.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.44020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48590.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44020.log.gz 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49716.log: /var/log/ceph/ceph-client.admin.48590.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49716.log.gz 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48590.log.gz 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50506.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49901.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50506.log.gz 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48176.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.49901.log: /var/log/ceph/ceph-client.admin.48176.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49901.log.gz 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48176.log.gz 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T11:03:50.214 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57989.log 2024-04-25T11:03:50.215 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.57989.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57989.log.gz 2024-04-25T11:03:50.215 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T11:03:50.215 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47770.log 2024-04-25T11:03:50.215 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47770.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47770.log.gz 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48375.log 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48375.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48375.log.gz 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50532.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47969.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T11:03:50.216 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-04-25T11:03:50.217 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2024-04-25T11:03:50.217 INFO:teuthology.orchestra.run.smithi155.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-04-25T11:03:50.218 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.43641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47891.log 2024-04-25T11:03:50.218 INFO:teuthology.orchestra.run.smithi155.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2024-04-25T11:03:50.219 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51275.log 2024-04-25T11:03:50.219 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47891.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47891.log.gz 2024-04-25T11:03:50.219 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.51275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58183.log 2024-04-25T11:03:50.219 INFO:teuthology.orchestra.run.smithi155.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51275.log.gz 2024-04-25T11:03:50.220 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:03:50.220 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58183.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58183.log.gz 2024-04-25T11:03:50.222 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.log: 94.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:03:50.222 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-04-25T11:03:50.222 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.43588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57045.log 2024-04-25T11:03:50.223 INFO:teuthology.orchestra.run.smithi155.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T11:03:50.223 INFO:teuthology.orchestra.run.smithi155.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-04-25T11:03:50.223 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47772.log 2024-04-25T11:03:50.224 INFO:teuthology.orchestra.run.smithi155.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57045.log.gz 2024-04-25T11:03:50.224 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57820.log 2024-04-25T11:03:50.224 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47772.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47772.log.gz 2024-04-25T11:03:50.224 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58874.log 2024-04-25T11:03:50.224 INFO:teuthology.orchestra.run.smithi155.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57820.log.gz 2024-04-25T11:03:50.225 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58874.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58874.log.gz 2024-04-25T11:03:50.225 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59022.log 2024-04-25T11:03:50.225 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57768.log 2024-04-25T11:03:50.226 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.59022.log: /var/log/ceph/ceph-client.admin.57768.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59022.log.gz 2024-04-25T11:03:50.227 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47771.log 2024-04-25T11:03:50.227 INFO:teuthology.orchestra.run.smithi155.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57768.log.gz 2024-04-25T11:03:50.227 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58975.log 2024-04-25T11:03:50.227 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47771.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47771.log.gz 2024-04-25T11:03:50.228 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-04-25T11:03:50.228 INFO:teuthology.orchestra.run.smithi155.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58975.log.gz 2024-04-25T11:03:50.228 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.43264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58722.log 2024-04-25T11:03:50.228 INFO:teuthology.orchestra.run.smithi155.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-04-25T11:03:50.229 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-04-25T11:03:50.229 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58722.log.gz 2024-04-25T11:03:50.229 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2024-04-25T11:03:50.229 INFO:teuthology.orchestra.run.smithi155.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-04-25T11:03:50.230 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50318.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2024-04-25T11:03:50.230 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50317.log 2024-04-25T11:03:50.230 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57819.log 2024-04-25T11:03:50.232 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50317.log: /var/log/ceph/ceph-client.admin.57819.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50317.log.gz 2024-04-25T11:03:50.232 INFO:teuthology.orchestra.run.smithi155.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57819.log.gz 2024-04-25T11:03:50.232 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50047.log 2024-04-25T11:03:50.233 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47968.log 2024-04-25T11:03:50.233 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50047.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50047.log.gz 2024-04-25T11:03:50.233 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47968.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47968.log.gz 2024-04-25T11:03:50.233 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58526.log 2024-04-25T11:03:50.233 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T11:03:50.233 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.58526.log: /var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58526.log.gz 2024-04-25T11:03:50.234 INFO:teuthology.orchestra.run.smithi155.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T11:03:50.235 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-04-25T11:03:50.235 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48100.log 2024-04-25T11:03:50.235 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48374.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-04-25T11:03:50.236 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48100.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48100.log.gz 2024-04-25T11:03:50.236 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48692.log 2024-04-25T11:03:50.236 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50838.log 2024-04-25T11:03:50.236 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.48692.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48692.log.gz 2024-04-25T11:03:50.237 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.50838.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50838.log.gz 2024-04-25T11:03:50.237 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57139.log 2024-04-25T11:03:50.237 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-04-25T11:03:50.238 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57139.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57139.log.gz 2024-04-25T11:03:50.239 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47499.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-04-25T11:03:50.239 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57448.log 2024-04-25T11:03:50.239 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57938.log 2024-04-25T11:03:50.239 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57448.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57448.log.gz 2024-04-25T11:03:50.240 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.57938.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57938.log.gz 2024-04-25T11:03:50.240 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47967.log 2024-04-25T11:03:50.240 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.47967.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47967.log.gz 2024-04-25T11:03:50.254 INFO:teuthology.orchestra.run.smithi155.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T11:03:50.273 INFO:teuthology.orchestra.run.smithi155.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T11:03:50.281 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T11:03:50.301 INFO:teuthology.orchestra.run.smithi155.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T11:03:50.336 INFO:teuthology.orchestra.run.smithi155.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T11:03:50.338 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-04-25T11:03:50.338 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m0.177s 2024-04-25T11:03:50.338 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m0.599s 2024-04-25T11:03:50.338 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.089s 2024-04-25T11:03:50.338 INFO:tasks.ceph:Archiving logs... 2024-04-25T11:03:50.339 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/remote/smithi155/log 2024-04-25T11:03:50.340 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:03:50.586 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:03:50.605 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:03:50.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:03:50.656 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:03:50.671 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:03:50.671 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:03:50.689 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2024-04-25T11:03:50.693 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:03:50.693 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-04-25T11:03:50.693 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 +1068us[+1068us] +/- 58ms 2024-04-25T11:03:50.693 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -350us[ -350us] +/- 65ms 2024-04-25T11:03:50.693 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -933us[ -933us] +/- 72ms 2024-04-25T11:03:50.693 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:03:50.694 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:03:50.705 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:03:50.706 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:03:50.717 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:03:50.798 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-25T11:03:50.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:03:50.827 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:03:50.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:03:50.852 INFO:teuthology.task.internal:Duration was 692.395258 seconds 2024-04-25T11:03:50.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:03:50.866 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:03:50.866 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:03:50.913 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:03:51.188 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:03:51.188 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T11:03:51.189 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:03:51.210 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:03:51.210 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:03:51.292 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:03:51.292 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:03:51.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:03:51.415 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:03:51.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:03:51.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:03:51.455 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:03:51.507 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-04-25T11:03:51.538 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:03:51.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:03:51.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:03:51.609 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:03:51.610 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673106/remote/smithi155 2024-04-25T11:03:51.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:03:51.675 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:03:51.675 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:03:51.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:03:51.741 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:03:51.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:03:51.754 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:03:51.754 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:03:51.783 INFO:teuthology.orchestra.run.smithi155.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:03 /home/ubuntu/cephtest 2024-04-25T11:03:51.785 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:03:51.800 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:03:51.831 INFO:teuthology.run:Summary data: description: fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-pacific 1-upgrade}} duration: 692.3952584266663 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:03:51.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:03:51.923 INFO:teuthology.run:pass